Alexandre Drouin

philippebrouillard@gmail.com

Doctorat - UdeM

Superviseur⋅e principal⋅e :

Irina Rish

arjun.ashok@mila.quebec

Doctorat - Polytechnique

Co-superviseur⋅e :

Quentin Cappart

leo.boisvert@mila.quebec

Philippe Brouillard

Doctorat - UdeM

Superviseur⋅e principal⋅e :

Dhanya Sridhar

Site web

Github

Google Scholar

Publications

Backdoor Decontamination Dynamics in LLM Agents

Christopher Pal

Open-weight LLM agents are vulnerable to backdoors installed during fine-tuning, which may be undetectable if the trigger conditions are nev… (voir plus)er met during testing. Assuming defenders do not know the existing trigger, they cannot unlearn it directly. One decontamination strategy is to install a known backdoor (defensive poisoning) then to unlearn it, hoping that the original unknown backdoor is removed as a side effect. However, this procedure has uncertain outcomes: the original backdoor may persist or be erased or rerouted, among other possibilities. We introduce a framework for studying these dynamics in tool-calling agents, decoupling trigger, response, teacher, and fine-tuning method across systematic experiments on AgentDyn. Across 115 experiments, defensive poisoning alone erases around 56% of original backdoors; subsequent decontamination then drives almost all survivors to erasure, confirming that trigger recognition and malicious execution are behaviorally dissociable. Interestingly, our experiments find that malicious backdoors never persist when using different triggers of the same general type as the defensive backdoor when followed by decontamination via unlearning. Co-installing up to four backdoors increases resistance (around 36% erased), yet decontaminating a single known co-resident backdoor collaterally clears 52/60 co-residents (87%). Upon visualizing postdecontamination model internals using J-lens, we confirm that although the decontamination restores benign LLM responses, traces of original trigger awareness persist at intermediate layers.

2026-08-10

arXiv (prépublication)

Invariant Causal Set Covering Machine

Thibaud Godon

Baptiste Bauvin

Pascal Germain

Jacques Corbeil

Rule-based models, such as decision trees, appeal to practitioners due to their interpretable nature. However, the learning algorithms that … (voir plus)produce such models are often vulnerable to spurious associations, and thus, they are not guaranteed to extract causally relevant insights. This limitation reduces their utility in gaining mechanistic insights into a phenomenon of interest. In this work, we build on ideas from the invariant causal prediction literature to propose Invariant Causal Set Covering Machines, an extension of the classical Set Covering Machine (SCM) algorithm for conjunctions/disjunctions of binary-valued rules that provably avoids spurious associations. The proposed method leverages structural assumptions about the functional form of such models, enabling an algorithm that identifies the causal parents of a variable of interest in polynomial time. We demonstrate the validity and efficiency of our approach through a simulation study and highlight its favorable performance compared to SCM in uncovering causal variables across real-world datasets.

2026-08-09

Transactions on Machine Learning Research (accepté)

Beyond Naïve Prompting: Strategies for Improved Context-aided Forecasting with LLMs

Andrew Robert Williams

Vincent Zhihao Zheng

Irina Rish

Nicolas Chapados

Étienne Marcotte

Valentina Zantedeschi

Real-world forecasting requires models to integrate not only historical data but also relevant contextual information provided in textual fo… (voir plus)rm. While large language models (LLMs) show promise for context-aided forecasting, critical challenges remain: we lack diagnostic tools to understand failure modes, performance remains far below their potential, and high computational costs limit practical deployment. We introduce a unified framework of four strategies that address these limitations along three orthogonal dimensions: model diagnostics, accuracy, and efficiency. Through extensive evaluation across model families from small open-source models to frontier models including Gemini, GPT, and Claude, we uncover both fundamental insights and practical solutions. Our findings span three key dimensions: diagnostic strategies reveal the “Execution Gap” where models correctly explain how context affects forecasts but fail to apply this reasoning; accuracy-focused strategies achieve substantial performance improvements of 25-50%; and efficiency-oriented approaches show that adaptive routing between small and large models can approach large model accuracy on average while significantly reducing inference costs. These orthogonal strategies can be flexibly integrated based on deployment constraints, providing practitioners with a comprehensive toolkit for practical LLM-based context-aided forecasting. Code is made available at https://github.com/ashok-arjun/beyond-naive-prompting.

2026-07-27

Transactions on Machine Learning Research (accepté)

Apriel-Reasoner: RL Post-Training for General-Purpose and Efficient Reasoning

Rafael Pardinas

Ehsan Kamalloo

David Vázquez

Building general-purpose reasoning models using reinforcement learning with verifiable rewards (RLVR) across diverse domains has been widely… (voir plus) adopted by frontier open-weight models. However, their training recipes and domain mixtures are often not disclosed. Joint optimization across domains poses significant challenges: domains vary widely in rollout length, problem difficulty, and sample efficiency. Further, models with long chain-of-thought traces increase inference cost and latency, making efficiency critical for practical deployment. We present Apriel-Reasoner, trained with a reproducible multi-domain RL post-training recipe on Apriel-Base, a 15B-parameter open-weight LLM, across five domains using public datasets: mathematics, code generation, instruction following, logical puzzles, and function calling. We introduce adaptive domain sampling that preserves target completed-rollout ratios despite heterogeneous rollout dynamics, and a difficulty-aware length penalty that, at no additional training overhead, encourages longer reasoning for difficult problems and shorter traces for easy ones. Trained with a strict 16K-token output budget, Apriel-Reasoner remains effective at a 32K output budget and improves over Apriel-Base on AIME 2025, GPQA, MMLU-Pro, and LiveCodeBench while producing 30-50% shorter reasoning traces. Among the evaluated open-weight models of similar scale, it improves the accuracy-token trade-off using a fully public-data recipe.

2026-07-07

Conference on Language Modeling (accepté)

AgentBeats: Agentifying Agent Assessment for Openness, Standardization, and Reproducibility

Xiaoyuan Liu

Jianhong Tu

Yuqi Chen

Siyuan Xie

Sihan Ren

Tianneng Shi

Gal Gantar

Evan Sandoval

D Lee

Daniel Miao

Peter J. Gilbert

Nick Hynes

Mauro Staver

Warren He

David Marn

Andrew Low

Xi Zhang

Elron Bandel

Michal Shmueli-Scheuer

Somasekhar Reddy … (voir 9 de plus)

Alexandre Lacoste

R Radha Krishnan

Elham Tabassi

Yu Su

Victor Barres

Chenguang Wang

Wenbo Guo

Dawn Song

Agent systems are advancing quickly across domains, but their evaluation remains fragmented. Most benchmarks rely on fixed, LLM-centric harn… (voir plus)esses that require heavy integration, create test-production mismatch, and limit fair comparison across diverse agent designs. The root problem is the lack of an open, agent-agnostic assessment interface. We advocate Agentified Agent Assessment (AAA), where evaluation is performed by judge agents and all participants interact through standardized protocols: A2A for task management and MCP for tool access. Conventional benchmarking defines two separate interfaces, one for the benchmark and one for the agent, while AAA only needs one; this yields a generic, unified framework that separates assessment logic from agent implementation and enables reproducible, interoperable, and multi-agent evaluation. We further introduce AgentBeats as a concrete realization of AAA: we identify five practical operation modes that make standardized assessment compatible with real-world constraints on openness, privacy, and reproducibility. To evaluate our design at scale, we conduct two studies: a five-month open competition that drew 298 judge agents across 12 categories together with 467 subject agents from independent participants, showing that AAA applies across a heterogeneous range of benchmarks; and a case study on coding agents that confirms agentified evaluation preserves fidelity with the public record while surfacing previously missing head-to-head results, yielding research insights about agent design. Combining a community-scale field study and a controlled coding case study, we verify that AAA delivers coverage, practicality, and fidelity across heterogeneous scenarios at scale. Together, AAA and AgentBeats offer a clear path toward open, standardized, and reproducible agent assessment.

2026-06-10

arXiv (prépublication)

Causal Discovery with Metadata-Informed Latent Types

Philippe Brouillard

Dhanya Sridhar

Causal discovery seeks to recover causal structure from data but the underlying graph is typically identifiable only up to its Markov equiva… (voir plus)lence class. Yet real-world systems often exhibit redundancy, where groups of variables share similar causal roles. We introduce a Bayesian causal discovery framework that leverages variable-level metadata to infer latent types and constrain causal interactions across variables. We model causal graphs as type-consistent DAGs and propose t-DiBS, a fully differentiable method that jointly learns variable types, graph structure, and metadata representations. Our approach enables principled uncertainty quantification and integrates expressive neural models for metadata. We provide theoretical results showing that, under structured assumptions, metadata combined with typing can improve identifiability beyond classical limits. Empirically, we demonstrate improved performance over standard causal discovery methods on synthetic and pseudo-real datasets, with detailed analysis demonstrating the benefit of joint type and structure learning. These results establish metadata-driven typing as a principled approach to identifiable causal discovery.

2026-05-30

Conference on Uncertainty in Artificial Intelligence (poster)

MosaicLeaks:Privacy Risks in Querying-in-the-Open for Deep Research Agents

Alexander Gurung

Spandana Gella

Issam H. Laradji

Perouz Taslakian

Rafael Pardinas

Deep research agents increasingly combine private local documents with external tools like web retrieval, creating a privacy risk: an agent'… (voir plus)s external queries may leak sensitive information from its local context. This risk is amplified by the mosaic effect, where individual queries may appear harmless but become revealing in aggregate. We introduce MosaicLeaks, a benchmark of 1,001 multi-hop deep research tasks that chain private enterprise documents and a public web corpus, forcing agents to make external queries that depend on local information. We evaluate leakage with an adversary LLM that observes only the agent's external queries and attempts to infer private information at three levels: the agent's research intent, answers to specific private questions and verifiable claims about the enterprise documents. We find that models across families and sizes frequently leak at all three levels, that zero-shot privacy prompting reduces but does not eliminate leakage and that reinforcement learning for task performance alone worsens leakage. To address this, we propose Privacy-Aware Deep Research (PA-DR), an RL framework that combines situational rewards for task success with a learned privacy classifier to provide dense credit assignment over both per-query and mosaic-level leakage. Training Qwen3-4B-Instruct with PA-DR improves accuracy from 48.7% to 58.7% and reduces answer and full-information leakage from 34.0% to 9.9%.

2026-05-28

arXiv (prépublication)

Dr-CiK: A Testbed for Foresight-Driven Agents

Yihong Tang

Andrew Williams

Vincent Zhihao Zheng

Lijun Sun

Issam H. Laradji

Étienne Marcotte

Valentina Zantedeschi

Time series forecasting in real-world settings often depends not only on historical observations, but also on external context that must be … (voir plus)actively discovered from noisy, heterogeneous information sources. Yet existing context-aided forecasting benchmarks typically assume that the supporting context is already provided, leaving open whether agents can identify it on their own. Therefore, we introduce Dr-CiK, a benchmark for evaluating whether agents can retrieve forecasting-relevant supporting context from a document corpus, filter out distractors, distill the retrieved context into forecast-useful evidence, and generate forecasts supported by that evidence. Through context ablations and evaluations of state-of-the-art deep research and forecasting methods paired together, we show that high-quality context substantially improves forecasting performance in Dr-CiK. However, most existing DR agents recover only a small fraction of the ground-truth supporting evidence (usually <5%), are frequently misled by distractors (>80% distractor citations), and can cause forecasters to perform worse with retrieved context than without context. Our results motivate research on foresight-driven agents that search for the right context to predict the future.

2026-05-26

arXiv (prépublication)

Malice in Agentland: Down the Rabbit Hole of Backdoors in the AI Supply Chain

Léo Boisvert

Abhay Puri

Chandra Kiran Reddy Evuru

Nazanin Sepahvand

Nicolas Chapados

Quentin Cappart

Alexandre Lacoste

Krishnamurthy (DJ) Dvijotham

The practice of fine-tuning AI agents on data from their own interactions--such as web browsing or tool use--, while being a strong general … (voir plus)recipe for improving agentic capabilities, also introduces a critical security vulnerability within the AI supply chain. In this work, we show that adversaries can easily poison the data collection pipeline to embed hard-to-detect backdoors that are triggerred by specific target phrases, such that when the agent encounters these triggers, it performs an unsafe or malicious action. We formalize and validate three realistic threat models targeting different layers of the supply chain: 1) direct poisoning of fine-tuning data, where an attacker controls a fraction of the training traces; 2) environmental poisoning, where malicious instructions are injected into webpages scraped or tools called while creating training data; and 3) supply chain poisoning, where a pre-backdoored base model is fine-tuned on clean data to improve its agentic capabilities. Our results are stark: by poisoning as few as 2% of the collected traces, an attacker can embed a backdoor causing an agent to leak confidential user information with over 80% success when a specific trigger is present. This vulnerability holds across all three threat models. Furthermore, we demonstrate that prominent safeguards, including two guardrail models and one weight-based defense, fail to detect or prevent the malicious behavior. These findings highlight an urgent threat to agentic AI development and underscore the critical need for rigorous security vetting of data collection processes and end-to-end model supply chains.

2026-05-25

ACM Conference on AI and Agentic Systems (publié)

WebArena-Pro: A Heterogeneous, Multimodal, Reproducible Benchmark for Web Agents

Imene Kerboua

Fatemeh Pesaran zadeh

Xing Han Lu

Weijian Qi

Alexander Miller

Junyi Song

Yunjia Tian

Dongjin Kang

Seyeon Choi

Marzia Nouri

Ewen Gueguen

Matteo Boglioni

Fengyuan Liu

Zeyi Liao

Mengqi Yuan

Yue Li

Alexandre Lacoste

Spandana Gella

Huan Sun … (voir 2 de plus)

Gunhee Kim

Siva Reddy

Web agents powered by large language and vision-language models are increasingly applied to realistic browser work that spans heterogeneous … (voir plus)applications, multimodal content, and stateful workflows. However, existing reproducible web-agent benchmarks cover only a small number of web applications drawn from a few software categories, and restrict modality to text and vision. Live benchmarks broaden site coverage but sacrifice reproducibility, since pages and data drift between runs. Moreover, existing benchmarks do not meaningfully evaluate whether agents can understand and use audio and video content embedded within web tasks. To address these gaps, we introduce WebArena-Pro, a benchmark comprising 300 tasks across 20 self-hosted web applications in six domain categories, spanning distinct interface conventions, workflows, and data models. Across the evaluated agents, the best performance is achieved by Gemini 3.1 Pro, which attains 37.0 % success under a 50-step budget, while open-source models' performance does not exceed 27.7% success. Among reproducible, human-curated web agent benchmarks, WebArena-Pro provides the broadest application coverage and the most comprehensive multimodal support to date. The benchmark treats audio and video as core observations alongside text and vision, with dedicated actions for extracting information from each. WebArena-Pro runs each task in isolation and supports reproducible, parallel evaluation. Tasks are authored through a dedicated annotator interface, filtered by LLM-assisted triage, and finally validated by humans before release.

2026-05-22

AIWILD @ International Conference on Machine Learning (publié)

Overcoming the Modality Gap in Context-Aided Forecasting

Vincent Zhihao Zheng

Étienne Marcotte

Andrew Robert Williams

Lijun Sun

Valentina Zantedeschi

Context-aided forecasting (CAF) holds promise for integrating domain knowledge and forward-looking information, enabling AI systems to surpa… (voir plus)ss traditional statistical methods. However, recent empirical studies reveal a puzzling gap: multimodal models often fail to outperform their unimodal counterparts. We hypothesize that this underperformance stems from poor context quality in existing datasets, as verification is challenging. To address these limitations, we introduce a semi-synthetic data augmentation method that generates contexts both descriptive of temporal dynamics and verifiably complementary to numerical histories. This approach enables massive-scale dataset creation, resulting in CAF-7M, a corpus of 7 million context-augmented time series windows, including a rigorously verified test set. We demonstrate that semi-synthetic pre-training transfers effectively to real-world evaluation, and show clear evidence of context utilization. Our results suggest that dataset quality, rather than architectural limitations, has been the primary bottleneck in context-aided forecasting.

2026-04-29

International Conference on Machine Learning (accepté)

Apriel-1.5-OpenReasoner: RL Post-Training for General-Purpose and Efficient Reasoning

Rafael Pardinas

Ehsan Kamalloo

David Vázquez

Building general-purpose reasoning models using reinforcement learning with verifiable rewards (RLVR) across diverse domains has been widely… (voir plus) adopted by frontier open-weight models. However, their training recipes and domain mixtures are often not disclosed. Joint optimization across domains poses significant challenges: domains vary widely in rollout length, problem difficulty and sample efficiency. Further, models with long chain-of-thought traces increase inference cost and latency, making efficiency critical for practical deployment. We present Apriel-1.5-OpenReasoner, trained with a fully reproducible multi-domain RL post-training recipe on Apriel-Base, a 15B-parameter open-weight LLM, across five domains using public datasets: mathematics, code generation, instruction following, logical puzzles and function calling. We introduce an adaptive domain sampling mechanism that preserves target domain ratios despite heterogeneous rollout dynamics, and a difficulty-aware extension of the standard length penalty that, with no additional training overhead, encourages longer reasoning for difficult problems and shorter traces for easy ones. Trained with a strict 16K-token output budget, Apriel-1.5-OpenReasoner generalizes to 32K tokens at inference and improves over Apriel-Base on AIME 2025, GPQA, MMLU-Pro, and LiveCodeBench while producing 30-50% shorter reasoning traces. It matches strong open-weight models of similar size at lower token cost, thereby pushing the Pareto frontier of accuracy versus token budget.

2026-04-01

arXiv (prépublication)