Portrait of Tomás Vergara Browne

Tomás Vergara Browne

tomasvergara.browne@mila.quebec
PhD - McGill University
Supervisor
Siva Reddy
Research Topics
Causality-Inspired Methods
Interpretability
Mechanistic Interpretability
Neural Scaling Laws
Website
Google Scholar
GitHub

Publications

AdaptArena: Evaluating Test-Time Personalization of Web Agents
Dongchan Shin
Xing Han Lu
Jiaqi Deng
Jay Gala
Tomás Vergara Browne
Jaewon Moon
Fengyuan Liu
Alexandre Drouin
Siva Reddy
Alexandre Lacoste
Large language model (LLM) agents have demonstrated strong performance on complex web navigation tasks, yet they remain brittle in real-worl… (see more)d settings where user intentions are underspecified and preferences are heterogeneous. In practice, users rarely provide explicit profiles, requiring agents to infer latent preferences from implicit signals. Despite its importance for deployment, this problem setting is largely underexplored in existing benchmarks. To address this gap, we introduce AdaptArena, a benchmark for evaluating test-time personalization of web agents via implicit preference inference. AdaptArena consists of 480 tasks, featuring both single-preference and double-preference scenarios. Each evaluation task must be solved by retrieving and leveraging the most relevant historical user trajectory that implicitly encodes the target preference. In addition, we introduce AdaptiveAgent, a retrieval-based framework for standardized evaluation of implicit preference inference. Experiments reveal a substantial performance gap: while oracle agents with access to ground-truth preferences achieve an 82.92% success rate, the evaluated LLM agents using our framework reach at most 15.62%. Furthermore, we find that correctly inferring user preferences is necessary but not sufficient for task success, as execution failures in downstream web interactions remain a significant bottleneck even when agents align with the target preference. These findings highlight implicit preference inference and robust action grounding as key challenges for deploying reliable, user-facing web agents. We release our code: https://github.com/McGill-NLP/web-agents-test-time-adaptations
2026-09-28
arXiv (preprint)
doi.org
arxiv.org
Operationalizing the Superficial Alignment Hypothesis via Task Complexity
Tomás Vergara Browne
Darshan Patil
Ivan Titov
Siva Reddy
Tiago Pimentel
Marius Mosbach
The superficial alignment hypothesis (SAH) posits that large language models learn most of their knowledge during pre-training, and that pos… (see more)t-training merely surfaces this knowledge. The SAH, however, lacks a precise definition, which has led to (i) different and seemingly orthogonal arguments supporting it, and (ii) important critiques to it. We propose a new metric called **Task Complexity**: the length of the shortest program that achieves a target performance on a task. In this framework, the SAH claims that pre-trained models drastically reduce the task complexity of achieving high performance on many tasks. Our definition unifies prior arguments supporting the SAH, interpreting them as different strategies to find such short programs. Experimentally, we estimate task complexities of mathematical reasoning, machine translation, and instruction following tasks and show that their respective task complexities can be remarkably low when conditioned on a pre-trained model. Further, we find that pre-training enables access to strong performances on our tasks, but it can require programs of gigabytes of length to access them. Post-training, on the other hand, collapses the complexity of reaching this same performance by several orders of magnitude. Overall, our results highlight that task adaptation can require remarkably little information—often just a few kilobytes.
2025-12-31
International Conference on Machine Learning (Accept (regular))
openreview.net
openreview.net