Christian Gagné

frederic.beaupre2@mila.quebec

Biography

Christian Gagné has been a professor in the Department of Electrical and Computer Engineering at Université Laval since 2008.

He is the director of the Institute Intelligence and Data (IID), holds a Canada CIFAR AI Chair, and is an associate member of Mila – Quebec Artificial Intelligence Institute.

Gagné is also a member of Université Laval’s Computer Vision and Systems Laboratory (LVSN), as well as its Robotics, Vision and Machine Intelligence Research Centre (CeRVIM) and its Big Data Research Centre (CRDM). He is a member of the REPARTI and UNIQUE strategic clusters of the FRQNT, the VITAM centre of the FRQS, and the International Observatory on the Societal Impacts of AI and Digital Technologies (OBVIA).

Gagné’s research focuses on the development of methods for machine learning and stochastic optimization. In particular, he is interested in deep neural networks, representation learning and transfer, meta-learning and multitasking. He is also interested in optimization approaches based on probabilistic models and evolutionary algorithms, including black-box optimization and automatic programming. An important part of his work is the practical application of these techniques in fields like computer vision, microscopy, healthcare, energy and transportation.

Current Students

Frédéric Beaupré

PhD - Université Laval

catherine.bouchard@mila.quebec

Catherine Bouchard

PhD - Université Laval

Olivier Bussière

Master's Research - Université Laval

olivier2202@gmail.com

Nizar El Ghazal

PhD - Université Laval

nizarelghazal@gmail.com

benjamin.leger@mila.quebec

Sara Karami

PhD - Université Laval

sara.karami.v@gmail.com

Rose Nayoung Kwon

PhD - Université Laval

nayoung.kwon@mila.quebec

Benjamin Léger

PhD - Université Laval

Samy Mammeri

Undergraduate - Université Laval

Co-supervisor :

Isabeau Prémont-Schwarz

samy.mammeri@mila.quebec

sabyasachi.sahoo@mila.quebec

Jonas Ngnawe

PhD - Université Laval

jonas.ngnawe@mila.quebec

PhD - Université Laval

Website

Adam Tupper

PhD - Université Laval

adam.tupper.1@ulaval.ca

Website

Publications

LLS: Learnable Layer Selection for Test-Time Adaptation

Suryansh Ram Menon

Frédéric Precioso

While Test-Time Adaptation (TTA) can recover performance drops caused by distribution shifts without requiring source labels, adapting all m… (see more)odel parameters frequently leads to catastrophic forgetting and overfitting to noisy test streams. Consequently, layer selection is critical. Existing methods such as GALA (Sahoo 2025) select layers via a cosine alignment proxy between gradient trajectories and parameter updates, a heuristic decoupled from the actual adaptation objective. We propose Learnable Layer Selection (LLS), a differentiable framework that learns continuous per-layer masks. The key insight of this work is not just to learn the layer selection masks, but also the idea to use gradients on one sample to learn the mask on another sample. By evaluating tentative parameter updates on a different, held-out validation sample, we introduce a novel self-supervised generalization loss. This forces the selection mechanism to learn layer masks that generalize to samples not explicitly optimized by current gradients, rather than overfitting. Evaluated on Full DomainBed (ResNet-18, 3 seeds, 16 environments), LLS achieves competitive accuracy across standard TTA algorithms, yielding the best target-split SHOT accuracy of 64.41 ± 0.96%. Systematic evaluations confirm that this cross-sample approach provides a highly stable mechanism for selective parameter adaptation.

2026-09-29

TTCL @ Neural Information Processing Systems (poster)

SNIP++: Fine-Grained Symbolic-Numerical Alignment for Symbolic Regression

Kazem Meidani

Mathematical expressions and the numerical behavior they produce are two views of the same underlying function, and connecting them is centr… (see more)al to scientific discovery. Symbolic Regression (SR) relies on this connection directly: it searches for an expression that reproduces a given behavior. Recent multi-modal models learn this connection by embedding symbolic expressions and their numerical behavior in a shared representation space. We show that this embedding space is only globally aligned: complete expressions correspond to complete behaviors, but the contribution of individual parts of an expression is not represented. This granularity gap leaves the model unable to tell how a local edit to an expression changes its behavior, the central operation in SR. We introduce a compositional alignment method that closes this gap: a structural positional encoding exposes the substructure of an expression to the encoder, and a multi-granularity contrastive objective grounds each subexpression in the behavior it produces before propagating this grounding to the full expression. The resulting representations close much of the modality gap between symbolic and numerical embeddings, reliably distinguish the effects of local edits that the original alignment cannot, and transfer to external SR corpora.

2026-09-24

arXiv (preprint)

Analyzing Flexible Search Distributions in Black-Box Optimization with Normalizing Flow-based Estimation of Distribution Algorithms

Sara Karami

Black-box optimization often requires search distributions that can adapt to complex geometric structures under limited evaluation budgets. … (see more)We propose NF-EDA, a Normalizing Flow-based Estimation of Distribution Algorithm that replaces fixed Gaussian models with a learned, flexible search distribution. Beyond optimization performance, our goal is to better understand how increased distributional expressiveness affects search behavior. In contrast to classical Gaussian-based methods, NF-EDA can adapt to curved, asymmetric, and non-elliptical regions of the search space, enabling broader yet structured exploration during early stages of optimization. By tracking the evolution of the learned distribution over iterations, we analyze how NF-EDA reshapes its sampling behavior compared to predefined parametric approaches such as CMA-ES and Gaussian EDAs. Experimental results on selected COCO BBOB functions, including Rastrigin, Schwefel, Lunacek bi-Rastrigin, and Rosenbrock, show that NF-EDA achieves faster early progress and reduced variability across runs, particularly in higher-dimensional settings. An ablation against a Gaussian EDA with matching update rules further demonstrates that these effects arise from the learned flow transformation rather than from the surrounding EDA procedure alone. These findings highlight the importance of flexible search distributions for understanding and improving model-based black-box optimization.

2026-08-12

Genetic and Evolutionary Computation Conference (published)

A Stochastic--Geometric Theory of Scaling Laws in Grokking

Róisín Luo

Jonas Ngnawe

Ihsan Ullah

Karyn Morrissey

Delayed generalization (\ie~grokking) refers to the phenomenon in which a neural network fits its training data early in training but only b… (see more)egins to generalize after a prolonged delay, often through an abrupt transition. Despite extensive empirical study, its underlying mechanism remains poorly understood. In this work, we first theoretically characterize a shell--core topological configuration of the reachable solution space induced by Adam's optimization dynamics with weight-shrinkage regularization, supported by empirical evidence. This optimization-induced topological configuration gives rise to grokking. In model's parameter space, random initialization solutions concentrate on a thin outer spherical shell, enclosing another spherical shell of memorization solutions, which in turn contains a core corresponding to the generalization solutions. Leveraging stopping-time theory, we then analyze the geometry of this topological configuration and the solution transition time at which optimization trajectories escape the memorization manifold and first reach the boundary of the generalization manifold. Our theoretical analysis derives grokking scaling laws for the learning rate, batch size, and

2026-06-28

arXiv (preprint)

Hidden-State Similarity Predicts Re-Elicitation After Inoculation Prompting

Fine-tuning on narrow harmful tasks can cause emergent misalignment, where models generalize harmful behavior beyond the training distributi… (see more)on. Inoculation prompting can reduce this effect by explicitly eliciting the undesired behavior during training, but recent work shows that the behavior can reappear when evaluation prompts contain cues from the training context. We study what makes such prompts effective triggers. We find that textual similarity to the inoculation prompt is an incomplete predictor: prompts are more likely to re-elicit suppressed behavior when they induce activation states similar to those produced by the inoculation context. These findings advance our understanding of how inoculation prompting modulates conditional misalignment, and suggest that activation-space analysis can help identify when suppressed behaviors remain accessible under eval-time prompts.

2026-06-10

ICML.cc/2026/Workshop/Mech_Interp (poster)

When Does Interleaving Prevent Emergent Misalignment?

Chen Sun

Large language models finetuned on narrow harmful tasks are prone to emergent misalignment (EM), where harmful behavior generalizes beyond t… (see more)he training distribution. Interleaving benign data during finetuning has been proposed as a mitigation, but recent work disagrees on whether it prevents EM. In this paper, we investigate this disagreement on Qwen-2.5 7B and 32B, and find that no single property of the interleaved data, taken in isolation, accounts for the gap. Instead, much of it traces to the evaluation itself, as the standard EM benchmark is sensitive to the length of the prompts it uses, and lengthening the evaluation prompts substantially shifts measured misalignment across model sizes. We then identify a region in the model's activations that predicts whether a given interleaved set will prevent EM, and show that reformulating benign data to fall within it substantially reduces EM on both 7B and 32B. This suggests that the standard EM benchmark, which relies on short prompts, may misrepresent the effectiveness of proposed mitigations.

2026-06-10

ICML.cc/2026/Workshop/Mech_Interp (poster)

Reliability-Gated Source Anchoring for Continual Test-Time Adaptation

Vikash Singh

Debargha Ganguly

Weicong Chen

Sabyasachi Sahoo

Sreehari Sankar

Biyao Zhang

Mohsen Hariri

Shouren Wang

Osama Zafar

Vipin Chaudhary

Continual test-time adaptation (CTTA) updates a pretrained model online on an unlabeled, non-stationary stream while anchoring it to a froze… (see more)n source checkpoint. This anchor is useful only when the source remains reliable. On CCC-Hard, however, a ResNet-50 source falls to approximately

2026-05-12

arXiv (preprint)

Multi-Modal Learning meets Genetic Programming: Analyzing Alignment in Latent Space Optimization

Benjamin Léger

Kazem Meidani

Symbolic regression (SR) aims to discover mathematical expressions from data, a task traditionally tackled using Genetic Programming (GP) th… (see more)rough combinatorial search over symbolic structures. Latent Space Optimization (LSO) methods use neural encoders to map symbolic expressions into continuous spaces, transforming the combinatorial search into continuous optimization. SNIP (Meidani et al., 2024), a contrastive pre-training model inspired by CLIP, advances LSO by introducing a multi-modal approach: aligning symbolic and numeric encoders in a shared latent space to learn the phenotype-genotype mapping, enabling optimization in the numeric space to implicitly guide symbolic search. However, this relies on fine-grained cross-modal alignment, whereas literature on similar models like CLIP reveals that such an alignment is typically coarse-grained. In this paper, we investigate whether SNIP delivers on its promise of effective bi-modal optimization for SR. Our experiments show that: (1) cross-modal alignment does not improve during optimization, even as fitness increases, and (2) the alignment learned by SNIP is too coarse to efficiently conduct principled search in the symbolic space. These findings reveal that while multi-modal LSO holds significant potential for SR, effective alignment-guided optimization remains unrealized in practice, highlighting fine-grained alignment as a critical direction for future work.

2026-04-08

arXiv (preprint)

Ca2+ transient detection and segmentation with the Astronomically motivated algorithm for Background Estimation And Transient Segmentation (Astro-BEATS)

Bi Fan

Anthony Bilodeau

Frédéric Beaupré

Theresa Wiesner

Flavie Lavoie-Cardinal

Renée Hložek

Fluorescence-based Ca…

2026-03-18

arXiv (preprint)

Robust Fine-Tuning from Non-Robust Pretrained Models: Mitigating Suboptimal Transfer With Epsilon-Scheduling

Ola Ahmad

Frédéric Precioso

Fine-tuning pretrained models is a standard and effective workflow in modern machine learning. However, robust fine-tuning (RFT), which aims… (see more) to simultaneously achieve adaptation to a downstream task and robustness to adversarial examples, remains challenging. Despite the abundance of non-robust pretrained models in open-source repositories, their potential for RFT is less understood. We address this knowledge gap by systematically examining RFT from such non-robust models. Our experiments reveal that fine-tuning non-robust models with a robust objective, even under small perturbations, can lead to poor performance, a phenomenon that we dub _suboptimal transfer_. In challenging scenarios (eg, difficult tasks, high perturbation), the resulting performance can be so low that it may be considered a transfer failure. We find that fine-tuning using a robust objective impedes task adaptation at the beginning of training and eventually prevents optimal transfer. However, we propose a novel heuristic, _Epsilon-Scheduling_, a schedule over perturbation strength used during training that promotes optimal transfer. Additionally, we introduce _expected robustness_, a metric that captures performance across a range of perturbations, providing a more comprehensive evaluation of the accuracy-robustness trade-off of diverse models at test-time. Extensive experiments on wide range of configurations (six pretrained models and five datasets) show that _Epsilon-Scheduling_ successfully prevents _suboptimal transfer_ and consistently improves expected robustness.

2025-12-31

International Conference on Learning Representations (Accept (Poster))

RobustGenBench: A Benchmark for Robust Generalization to Adversarial and Common Perturbations, with Applications to Vision and Vision-Enabled Large Language Models

Ola Ahmad

2025-12-31

Neural Information Processing Systems (Accept (poster))