Portrait de Fengyuan Liu

Fengyuan Liu

fengyuan.liu@mila.quebec
Maîtrise recherche - McGill
Superviseur⋅e principal⋅e
Siva Reddy
Co-supervisor
Dzmitry Bahdanau
Sujets de recherche
Agent basé sur un LLM
Alignement de l'IA
Apprentissage continu
Apprentissage décentralisé
Apprentissage fédéré
Apprentissage multimodal
Apprentissage par transfert
Apprentissage profond
Apprentissage sur variétés
Compression de modèles
Grands modèles de langage (LLM)
IAG (Intelligence Artificielle Générale)
Optimisation
Optimisation décentralisée
Traitement du langage naturel
XAI (IA explicable)
Site web
GitHub

Publications

AdaptArena: Evaluating Test-Time Personalization of Web Agents
Dongchan Shin
Xing Han Lu
Jiaqi Deng
Jay Gala
Tomás Vergara Browne
Jaewon Moon
Fengyuan Liu
Alexandre Drouin
Siva Reddy
Alexandre Lacoste
Large language model (LLM) agents have demonstrated strong performance on complex web navigation tasks, yet they remain brittle in real-worl… (voir plus)d settings where user intentions are underspecified and preferences are heterogeneous. In practice, users rarely provide explicit profiles, requiring agents to infer latent preferences from implicit signals. Despite its importance for deployment, this problem setting is largely underexplored in existing benchmarks. To address this gap, we introduce AdaptArena, a benchmark for evaluating test-time personalization of web agents via implicit preference inference. AdaptArena consists of 480 tasks, featuring both single-preference and double-preference scenarios. Each evaluation task must be solved by retrieving and leveraging the most relevant historical user trajectory that implicitly encodes the target preference. In addition, we introduce AdaptiveAgent, a retrieval-based framework for standardized evaluation of implicit preference inference. Experiments reveal a substantial performance gap: while oracle agents with access to ground-truth preferences achieve an 82.92% success rate, the evaluated LLM agents using our framework reach at most 15.62%. Furthermore, we find that correctly inferring user preferences is necessary but not sufficient for task success, as execution failures in downstream web interactions remain a significant bottleneck even when agents align with the target preference. These findings highlight implicit preference inference and robust action grounding as key challenges for deploying reliable, user-facing web agents. We release our code: https://github.com/McGill-NLP/web-agents-test-time-adaptations
2026-09-28
arXiv (prépublication)
doi.org
arxiv.org
WebArena-Pro: A Heterogeneous, Multimodal, Reproducible Benchmark for Web Agents
Imene Kerboua
Fatemeh Pesaran zadeh
Xing Han Lu
Weijian Qi
Alexander Miller
Junyi Song
Yunjia Tian
Dongjin Kang
Seyeon Choi
Marzia Nouri
Ewen Gueguen
Matteo Boglioni
Fengyuan Liu
Zeyi Liao
Mengqi Yuan
Yue Li
Alexandre Lacoste
Alexandre Drouin
Spandana Gella
Huan Sun … (voir 2 de plus)
Gunhee Kim
Siva Reddy
Web agents powered by large language and vision-language models are increasingly applied to realistic browser work that spans heterogeneous … (voir plus)applications, multimodal content, and stateful workflows. However, existing reproducible web-agent benchmarks cover only a small number of web applications drawn from a few software categories, and restrict modality to text and vision. Live benchmarks broaden site coverage but sacrifice reproducibility, since pages and data drift between runs. Moreover, existing benchmarks do not meaningfully evaluate whether agents can understand and use audio and video content embedded within web tasks. To address these gaps, we introduce WebArena-Pro, a benchmark comprising 300 tasks across 20 self-hosted web applications in six domain categories, spanning distinct interface conventions, workflows, and data models. Across the evaluated agents, the best performance is achieved by Gemini 3.1 Pro, which attains 37.0 % success under a 50-step budget, while open-source models' performance does not exceed 27.7% success. Among reproducible, human-curated web agent benchmarks, WebArena-Pro provides the broadest application coverage and the most comprehensive multimodal support to date. The benchmark treats audio and video as core observations alongside text and vision, with dedicated actions for extracting information from each. WebArena-Pro runs each task in isolation and supports reproducible, parallel evaluation. Tasks are authored through a dedicated annotator interface, filtered by LLM-assisted triage, and finally validated by humans before release.
2026-05-22
AIWILD @ International Conference on Machine Learning (publié)
openreview.net
openreview.net
BRIDGE: Predicting Human Task Completion Time From Model Performance
Fengyuan Liu
Jay Gala
Nilaksh
Dzmitry Bahdanau
Siva Reddy
Hugo Larochelle
Evaluating the real-world capabilities of AI systems requires grounding benchmark performance in human-interpretable measures of task diffic… (voir plus)ulty. Existing approaches that rely on direct human task completion time annotations are costly, noisy, and difficult to scale across benchmarks. In this work, we propose BRIDGE, a unified psychometric framework that learns the latent difficulty scale from model responses and anchors it to human task completion time. Using a two-parameter logistic Item Response Theory model, we jointly estimate latent task difficulty and model capability from model performance data across multiple benchmarks. We demonstrate that latent task difficulty varies linearly with the logarithm of human completion time, allowing human task completion time to be inferred for new benchmarks from model performance alone. Leveraging this alignment, we forecast frontier model capabilities in terms of human task length and independently reproduce METR's exponential scaling results, with the 50% solvable task horizon doubling approximately every 6 months.
2025-12-31
International Conference on Machine Learning (Accept (regular))
doi.org
proceedings.mlr.press