Audrey Durand

julien.armand@mila.quebec

Biographie

Audrey Durand est professeure adjointe au Département d’informatique et de génie logiciel ainsi qu’au Département de génie électrique et de génie informatique de l’Université Laval. Elle se spécialise dans les algorithmes qui apprennent par l’interaction avec leur environnement, soit l’apprentissage par renforcement, et s’intéresse particulièrement à l’application de ces approches au domaine de la santé.

Étudiants actuels

Julien Armand

Maîtrise recherche - Université Laval

Doctorat - McGill

Maîtrise recherche - Université Laval

baptiste.bonin@mila.quebec

Hugo Delhaye

Maîtrise recherche - UdeM

Superviseur⋅e principal⋅e :

Lune Bellec

hugo.delhaye@mila.quebec

Erinn Grigsby

Postdoctorat - UdeM

erinn.grigsby@mila.quebec

Site web

Github

alexandre.larouche@mila.quebec

Bruno Joyal

Maîtrise recherche - Université Laval

bruno.joyal@mila.quebec

Alexandre Larouche

Doctorat - Université Laval

Github

randy.lefebvre@mila.quebec

Randy Lefebvre

Doctorat - Université Laval

Hadi Moazen

Doctorat - Université Laval

hadi.moazen@mila.quebec

Caro Strickland

Postdoctorat - Université Laval

caro.strickland@mila.quebec

Site web

Github

Publications

A Guide to Robust Generalization: The Impact of Architecture, Pre-training, and Optimization Strategy

Ola Ahmad

Deep learning models operating in the image domain are vulnerable to small input perturbations. For years, robustness to such perturbations … (voir plus)was pursued by training models from scratch (i.e., with random initializations) using specialized loss objectives. Recently, robust fine-tuning has emerged as a more efficient alternative: instead of training from scratch, pretrained models are adapted to maximize predictive performance and robustness. To conduct robust fine-tuning, practitioners design an optimization strategy that includes the model update protocol (e.g., full or partial) and the specialized loss objective. Additional design choices include the architecture type and size, and the pretrained representation. These design choices affect robust generalization, which is the model's ability to maintain performance when exposed to new and unseen perturbations at test time. Understanding how these design choices influence generalization remains an open question with significant practical implications. In response, we present an empirical study spanning 6 datasets, 40 pretrained architectures, 2 specialized losses, and 3 adaptation protocols, yielding 1,440 training configurations and 7,200 robustness measurements across five perturbation types. To our knowledge, this is the most diverse and comprehensive benchmark of robust fine-tuning to date. While attention-based architectures and robust pretrained representations are increasingly popular, we find that convolutional neural networks pretrained in a supervised manner on large datasets often perform best. Our analysis both confirms and challenges prior design assumptions, highlighting promising research directions and offering practical guidance.

2025-09-28

NeurIPS.cc/2025/Workshop/Reliable_ML (publié)

On the Fundamental Limitations of Dual Static CVaR Decompositions in Markov Decision Processes

Mathieu Godbout

2025-07-17

ArXiv (prépublication)

arxiv.org

Multi-Agent Matrix Games with Individual learners: How Exploration-Exploitation Strategies Impact the Emergence of Coordination

Julien Armand

Tommy Chien-Hsuan Lin

Maxime Heuillet

Coordination between independent learning agents in a multi-agent environment is an important problem where AI systems may impact each other… (voir plus)s learning process. In this paper, we study how individual agents converge to optimal equilibrium in multi-agent where coordination is necessary to achieve optimality. Specifically, we cover the case of coordination to maximize every individual payoffs and coordination to maximize the collective payoff (cooperation). We study the emergence of such coordination behaviours in two-players matrix games with unknown payoff matrices and noisy bandit feedback. We consider five different environments along with widely used deterministic and stochastic bandit strategies. We study how different learning strategies and observation noise influence convergence to the optimal equilibrium. Our results indicate that coordination often emerge more easily from interactions between deterministic agents, especially when they follow the same learning behaviour. However, stochastic learning strategies appear to be more robust in the presence of many optimal joint actions. Overall, noisy observations often help stabilizing learning behaviours.

2025-06-22

rl-conference.cc/RLC/2025/Workshop/CoCoMARL (poster)

Human-AI Alignment of Learning Trajectories in Video Games: a continual RL benchmark proposal

Yann Harel

Lune P Bellec

François Paugam

Hugo Delhaye

We propose a design for a continual reinforcement learning (CRL) benchmark called GHAIA, centered on human-AI alignment of learning trajecto… (voir plus)ries in structured video game environments. Using \textit{Super Mario Bros.} as a case study, gameplay is decomposed into short, annotated scenes organized into diverse task sequences based on gameplay patterns and difficulty. Evaluation protocols measure both plasticity and stability, with flexible revisit and pacing schedules. A key innovation is the inclusion of high-resolution human gameplay data collected under controlled conditions, enabling direct comparison of human and agent learning. In addition to adapting classical CRL metrics like forgetting and backward transfer, we introduce semantic transfer metrics capturing learning over groups of scenes sharing similar game patterns. We demonstrate the feasibility of our approach on human and agent data, and discuss key aspects of the first release for community input.

2025-06-19

rl-conference.cc/RLC/2025/Workshop/RLVG (accepté)

Optimal discounting for offline input-driven MDP

Randy Lefebvre

Offline reinforcement learning has gained a lot of popularity for its potential to solve industry challenges. However, real-world environmen… (voir plus)ts are often highly stochastic and partially observable, leading long-term planners to overfit to offline data in model-based settings. Input-driven Markov Decision Processes (IDMDPs) offer a way to work with some of the uncertainty by letting designers separate what the agent has control over (states) from what it cannot (inputs) in the environnement. These stochastic external inputs are often difficult to model. Under the assumption that the input model will be imperfect, we investigate the bias-variance tradeoff under shallow planning in IDMDPs. Paving the way to input-driven planning horizons, we also investigate the similarity of optimal planning horizons at different inputs given the structure of the input space.

2025-05-08

rl-conference.cc/RLC/2025/Conference (accepté)

Platform-based Adaptive Experimental Research in Education: Lessons Learned from The Digital Learning Challenge

Ilya Musabirov

Mohi Reza

Haochen Song

Steven Moore

Pan Chen

Harsh Kumar

Tong Li

John Stamper

Norman Bier

Anna Rafferty

Thomas Price

Nina Deliu

Michael Liut

Joseph Jay Williams

: We report on our experience with a real-world, multi-experimental evaluation of an adaptive experimentation platform within the XPRIZE Dig… (voir plus)ital Learning Challenge framework. We showcase how EASI (Experiment as a Service) cross-platform software supports quick integration and deployment of adaptive experiments as well as five systematic replications within a 30-day timeframe. The outline the key scenarios of the applicability of platform-supported experiments and reflect on lessons learned from this two-year project that can help researchers and practitioners to integrate adaptive experiments in real-world courses

2025-03-02

Proceedings of the 15th International Learning Analytics and Knowledge Conference (publié)

Adaptive Experiments Under Data Sparse Settings: Applications for Educational Platforms

Haochen Song

Ilya Musabirov

Ananya Bhattacharjee

Meredith Franklin

Anna Rafferty

Joseph Jay Williams

Adaptive experimentation is increasingly used in educational platforms to personalize learning through dynamic content and feedback. However… (voir plus), standard adaptive strategies such as Thompson Sampling often underperform in real-world educational settings where content variations are numerous and student participation is limited, resulting in sparse data. In particular, Thompson Sampling can lead to imbalanced content allocation and delayed convergence on which aspects of content are most effective for student learning. To address these challenges, we introduce Weighted Allocation Probability Adjusted Thompson Sampling (WAPTS), an algorithm that refines the sampling strategy to improve content-related decision-making in data-sparse environments. WAPTS is guided by the principle of lenient regret, allowing near-optimal allocations to accelerate learning while still exploring promising content. We evaluate WAPTS in a learnersourcing scenario where students rate peer-generated learning materials, and demonstrate that it enables earlier and more reliable identification of promising treatments.

2025-01-06

ArXiv (prépublication)

arxiv.org

Robust Fine-Tuning from Non-Robust Pretrained Models: Mitigating Suboptimal Transfer With Adversarial Scheduling

Ola Ahmad

Frédéric Precioso

Fine-tuning pretrained models is a standard and effective workflow in modern machine learning. However, robust fine-tuning (RFT), which aims… (voir plus) to simultaneously achieve adaptation to a downstream task and robustness to adversarial examples, remains challenging. Despite the abundance of non-robust pretrained models in open-source repositories, their potential for RFT is less understood. We address this knowledge gap by systematically examining RFT from such non-robust models. Our experiments reveal that fine-tuning non-robust models with a robust objective, even under small perturbations, can lead to poor performance, a phenomenon that we dub \emph{suboptimal transfer}. In challenging scenarios (eg, difficult tasks, high perturbation), the resulting performance can be so low that it may be considered a transfer failure. We find that fine-tuning using a robust objective impedes task adaptation at the beginning of training and eventually prevents optimal transfer. However, we propose a novel heuristic, \emph{Epsilon-Scheduling}, a schedule over perturbation strength used during training that promotes optimal transfer. Additionally, we introduce \emph{expected robustness}, a metric that captures performance across a range of perturbations, providing a more comprehensive evaluation of the accuracy-robustness trade-off for diverse models at test time. Extensive experiments on a wide range of configurations (six pretrained models and five datasets) show that \emph{Epsilon-Scheduling} successfully prevents \emph{suboptimal transfer} and consistently improves expected robustness.

2024-12-31

arXiv (prépublication)

arxiv.org

Neural Active Learning Meets the Partial Monitoring Framework

Maxime Heuillet

Ola Ahmad

2024-09-11

Proceedings of the Fortieth Conference on Uncertainty in Artificial Intelligence (publié)

proceedings.mlr.press

Randomized Confidence Bounds for Stochastic Partial Monitoring

Maxime Heuillet

Ola Ahmad

The partial monitoring (PM) framework provides a theoretical formulation of sequential learning problems with incomplete feedback. On each r… (voir plus)ound, a learning agent plays an action while the environment simultaneously chooses an outcome. The agent then observes a feedback signal that is only partially informative about the (unobserved) outcome. The agent leverages the received feedback signals to select actions that minimize the (unobserved) cumulative loss. In contextual PM, the outcomes depend on some side information that is observable by the agent before selecting the action on each round. In this paper, we consider the contextual and non-contextual PM settings with stochastic outcomes. We introduce a new class of PM strategies based on the randomization of deterministic confidence bounds. We also extend regret guarantees to settings where existing stochastic strategies are not applicable. Our experiments show that the proposed RandCBP and RandCBPsidestar strategies have favorable performance against state-of-the-art baselines in multiple PM games. To advocate for the adoption of the PM framework, we design a use case on the real-world problem of monitoring the error rate of any deployed classification system.

2024-07-22

International Conference on Machine Learning (Accept (Poster))

proceedings.mlr.press

Data harmonization for Advancing research on Personalized Rehabilitation Interventions for Patients with Traumatic Brain Injury and Stroke: A proof of concept

Dorra Rakia Allegue

Despoina Petsani

Nathalie Ponthon

Evdokimos Konstantinidis

Panagiotis Bamidis

Eva Kehayia

Sara Ahmed

Stroke and traumatic brain injury (TBI) are leading causes of morbidity and mortality, affecting survivors’ mobility and social participat… (voir plus)ion. Although personalized interventions could positively impact survivors' recovery, the effectiveness of such interventions remains unclear. Open-access data repositories can provide access to multiple shared data which could help uncover new evidence of effective interventions; however, harmonizing data between different studies requires many steps to make it possible given the various methods of data collection, intervention characteristics and population sociodemographic profile. This proof-of-concept study aimed to describe the steps and anchors that contributed to the development of guiding frameworks to harmonize data across different studies. Data were extracted from the Federal Interagency Traumatic Brain Injury Research (FITBIR) repository and stored on an online cloud platform. The outcome measures were mapped to mobility determinants using the International Classification of Functioning, Disability, and Health (ICF) and Webber framework. The intervention's effect was categorized according to the Minimal Clinically Important Difference (MCID)s of the measures administered. The study proposed a novel framework for intervention features, which aims to enhance our understanding of the mechanisms of action and potential impact of rehabilitation interventions. The framework classified interventions based on their nature, context, specific body systems, dosage, caregiver assistance, and behaviour change strategies. In conclusion, this study demonstrated the feasibility of harmonizing data extracted from different sources in the FITBIR repository. Leveraging existing open databases offers tremendous opportunities to advance research on personalized interventions for patients with TBI and stroke and inform decision-making during transitions.

2024-06-25

Petra (publié)

On shallow planning under partial observability

Randy Lefebvre

2024-05-30

rl-conference.cc/RLC/2024/Workshop/Deployable_RL (publié)