Portrait de Maxime Heuillet n'est pas disponible

Maxime Heuillet

Doctorat - Université Laval
Superviseur⋅e principal⋅e
Audrey Durand
Sujets de recherche
Apprentissage par renforcement

Publications

Robust Fine-Tuning from Non-Robust Pretrained Models: Mitigating Suboptimal Transfer With Adversarial Scheduling
Jonas Ngnawe
Maxime Heuillet
Sabyasachi Sahoo
Yann Batiste Pequignot
Ola Ahmad
Audrey Durand
Frederic Precioso
Christian Gagné
Fine-tuning pretrained models is a standard and effective workflow in modern machine learning. However, robust fine-tuning (RFT), which aims… (voir plus) to simultaneously achieve adaptation to a downstream task and robustness to adversarial examples, remains challenging. Despite the abundance of non-robust pretrained models in open-source repositories, their potential for RFT is less understood. We address this knowledge gap by systematically examining RFT from such non-robust models. Our experiments reveal that fine-tuning non-robust models with a robust objective, even under small perturbations, can lead to poor performance, a phenomenon that we dub \emph{suboptimal transfer}. In challenging scenarios (eg, difficult tasks, high perturbation), the resulting performance can be so low that it may be considered a transfer failure. We find that fine-tuning using a robust objective impedes task adaptation at the beginning of training and eventually prevents optimal transfer. However, we propose a novel heuristic, \emph{Epsilon-Scheduling}, a schedule over perturbation strength used during training that promotes optimal transfer. Additionally, we introduce \emph{expected robustness}, a metric that captures performance across a range of perturbations, providing a more comprehensive evaluation of the accuracy-robustness trade-off for diverse models at test time. Extensive experiments on a wide range of configurations (six pretrained models and five datasets) show that \emph{Epsilon-Scheduling} successfully prevents \emph{suboptimal transfer} and consistently improves expected robustness.
2025-09-27
ArXiv (prépublication)
arxiv.org
arxiv.org
Nested-ReFT: Efficient Reinforcement Learning for Large Language Model Fine-Tuning via Off-Policy Rollouts
Maxime Heuillet
Yufei Cui
Boxing Chen
Audrey Durand
Prasanna Parthasarathi
2025-08-13
ArXiv (prépublication)
doi.org
arxiv.org
A Guide to Robust Generalization: The Impact of Architecture, Pre-training, and Optimization Strategy
Maxime Heuillet
Rishika Bhagwatkar
Jonas Ngnawe
Yann Batiste Pequignot
Alexandre Larouche
Christian Gagné
Irina Rish
Ola Ahmad
Audrey Durand
Deep learning models operating in the image domain are vulnerable to small input perturbations. For years, robustness to such perturbations … (voir plus)was pursued by training models from scratch (i.e., with random initializations) using specialized loss objectives. Recently, robust fine-tuning has emerged as a more efficient alternative: instead of training from scratch, pretrained models are adapted to maximize predictive performance and robustness. To conduct robust fine-tuning, practitioners design an optimization strategy that includes the model update protocol (e.g., full or partial) and the specialized loss objective. Additional design choices include the architecture type and size, and the pretrained representation. These design choices affect robust generalization, which is the model's ability to maintain performance when exposed to new and unseen perturbations at test time. Understanding how these design choices influence generalization remains an open question with significant practical implications. In response, we present an empirical study spanning 6 datasets, 40 pretrained architectures, 2 specialized losses, and 3 adaptation protocols, yielding 1,440 training configurations and 7,200 robustness measurements across five perturbation types. To our knowledge, this is the most diverse and comprehensive benchmark of robust fine-tuning to date. While attention-based architectures and robust pretrained representations are increasingly popular, we find that convolutional neural networks pretrained in a supervised manner on large datasets often perform best. Our analysis both confirms and challenges prior design assumptions, highlighting promising research directions and offering practical guidance.
2025-08-12
ArXiv (prépublication)
doi.org
arxiv.org
A Guide to Robust Generalization: The Impact of Architecture, Pre-training, and Optimization Strategy
Maxime Heuillet
Rishika Bhagwatkar
Jonas Ngnawe
Yann Batiste Pequignot
Alexandre Larouche
Christian Gagné
Irina Rish
Ola Ahmad
Audrey Durand
Deep learning models operating in the image domain are vulnerable to small input perturbations. For years, robustness to such perturbations … (voir plus)was pursued by training models from scratch (i.e., with random initializations) using specialized loss objectives. Recently, robust fine-tuning has emerged as a more efficient alternative: instead of training from scratch, pretrained models are adapted to maximize predictive performance and robustness. To conduct robust fine-tuning, practitioners design an optimization strategy that includes the model update protocol (e.g., full or partial) and the specialized loss objective. Additional design choices include the architecture type and size, and the pretrained representation. These design choices affect robust generalization, which is the model's ability to maintain performance when exposed to new and unseen perturbations at test time. Understanding how these design choices influence generalization remains an open question with significant practical implications. In response, we present an empirical study spanning 6 datasets, 40 pretrained architectures, 2 specialized losses, and 3 adaptation protocols, yielding 1,440 training configurations and 7,200 robustness measurements across five perturbation types. To our knowledge, this is the most diverse and comprehensive benchmark of robust fine-tuning to date. While attention-based architectures and robust pretrained representations are increasingly popular, we find that convolutional neural networks pretrained in a supervised manner on large datasets often perform best. Our analysis both confirms and challenges prior design assumptions, highlighting promising research directions and offering practical guidance.
2025-08-12
ArXiv (prépublication)
doi.org
arxiv.org
Multi-Agent Matrix Games with Individual learners: How Exploration-Exploitation Strategies Impact the Emergence of Coordination
Julien Armand
Tommy Chien-Hsuan Lin
Maxime Heuillet
Audrey Durand
Coordination between independent learning agents in a multi-agent environment is an important problem where AI systems may impact each other… (voir plus)s learning process. In this paper, we study how individual agents converge to optimal equilibrium in multi-agent where coordination is necessary to achieve optimality. Specifically, we cover the case of coordination to maximize every individual payoffs and coordination to maximize the collective payoff (cooperation). We study the emergence of such coordination behaviours in two-players matrix games with unknown payoff matrices and noisy bandit feedback. We consider five different environments along with widely used deterministic and stochastic bandit strategies. We study how different learning strategies and observation noise influence convergence to the optimal equilibrium. Our results indicate that coordination often emerge more easily from interactions between deterministic agents, especially when they follow the same learning behaviour. However, stochastic learning strategies appear to be more robust in the presence of many optimal joint actions. Overall, noisy observations often help stabilizing learning behaviours.
2025-06-23
rl-conference.cc/RLC/2025/Workshop/CoCoMARL (poster)
openreview.net
openreview.net
Randomized Confidence Bounds for Stochastic Partial Monitoring
Maxime Heuillet
Ola Ahmad
Audrey Durand
2024-07-08
Proceedings of the 41st International Conference on Machine Learning (publié)
doi.org
openreview.net
Neural Active Learning Meets the Partial Monitoring Framework
Maxime Heuillet
Ola Ahmad
Audrey Durand
We focus on the online-based active learning (OAL) setting where an agent operates over a stream of observations and trades-off between the … (voir plus)costly acquisition of information (labelled observations) and the cost of prediction errors. We propose a novel foundation for OAL tasks based on partial monitoring, a theoretical framework specialized in online learning from partially informative actions. We show that previously studied binary and multi-class OAL tasks are instances of partial monitoring. We expand the real-world potential of OAL by introducing a new class of cost-sensitive OAL tasks. We propose NeuralCBP, the first PM strategy that accounts for predictive uncertainty with deep neural networks. Our extensive empirical evaluation on open source datasets shows that NeuralCBP has favorable performance against state-of-the-art baselines on multiple binary, multi-class and cost-sensitive OAL tasks.
2024-04-26
auai.org/UAI/2024/Conference (poster)
doi.org
openreview.net
Neural Active Learning Meets the Partial Monitoring Framework
Maxime Heuillet
Ola Ahmad
Audrey Durand
2024-04-26
auai.org/UAI/2024/Conference (poster)
doi.org
openreview.net
Randomized Confidence Bounds for Stochastic Partial Monitoring
Maxime Heuillet
Ola Ahmad
Audrey Durand
2024-02-07
ArXiv (prépublication)
doi.org
arxiv.org