Sarath Chandar

istabrak.abbes@mila.quebec

Biographie

Sarath Chandar est professeur associé au départment de génie informatique et génie logiciel de Polytechnique Montréal, où il dirige le laboratoire de recherche Chandar. Il est également membre académique principal à Mila – Institut québécois d’intelligence artificielle, et titulaire d'une chaire en IA Canada-CIFAR et d'une Chaire de recherche du Canada en apprentissage machine permanent.

Ses recherches portent sur l'apprentissage tout au long de la vie, l'apprentissage profond, l'optimisation, l'apprentissage par renforcement et le traitement du langage naturel. Pour promouvoir la recherche sur l'apprentissage tout au long de la vie, Sarath Chandar a créé la Conférence sur les agents d'apprentissage tout au long de la vie (CoLLAs) en 2022 et a présidé le programme en 2022 et en 2023. Il est titulaire d'un doctorat de l'Université de Montréal et d'une maîtrise en recherche de l'Indian Institute of Technology Madras.

Étudiants actuels

Istabrak Abbes

Doctorat - UdeM

Alex Aselstyne

Maîtrise recherche - Polytechnique

alexander.aselstyne@mila.quebec

davide.baldelli@mila.quebec

Davide Baldelli

Doctorat - Polytechnique

Co-superviseur⋅e :

Amal Zouaq

prashant.govindarajan@mila.quebec

Milan Bhan

Collaborateur·rice de recherche

milan.bhan@mila.quebec

Prashant Govindarajan

Doctorat - Polytechnique

nathanael.haas@mila.quebec

Simon Guiroy

Doctorat - UdeM

Superviseur⋅e principal⋅e :

Stagiaire de recherche - Polytechnique

Maryam Hashemzadeh

Doctorat - UdeM

maryam.hashemzadeh@mila.quebec

david.heurtel-depeiges@mila.quebec

David Heurtel--Depeiges

Doctorat - Polytechnique

Jerry Huang

Doctorat - UdeM

jerry.huang@mila.quebec

Kanishk Jain

Doctorat - UdeM

Co-superviseur⋅e :

Ross Goroshin

kanishk.jain@mila.quebec

Saurav Jha

Postdoctorat - Polytechnique

saurav.jha@mila.quebec

Amir Ardalan Kalantari Dehaghi

Doctorat - UdeM

Visiteur de recherche indépendant - Technical University of Munich

F.kapl@web.de

Antoine Lafrance

Maîtrise recherche - Polytechnique

antoine.lafrance@mila.quebec

lola.lebreton@mila.quebec

Lola Le Breton

Doctorat - Polytechnique

Tom Lee

Postdoctorat - Polytechnique

thomas.lee@mila.quebec

ekaterina.lobacheva@mila.quebec

Aidan Li

Maîtrise recherche - UdeM

Co-superviseur⋅e :

Postdoctorat - UdeM

Doctorat - Polytechnique

mansi.maheshwari@mila.quebec

Roshan Munirathinam Sankaran Balaji

Collaborateur·rice de recherche - Polytechnique

roshan.munirathinam@mila.quebec

nilaksh.nilaksh@mila.quebec

Hadi NekoeiQachkanloo

Doctorat - UdeM

nekoeihe@mila.quebec

Nilaksh Nilaksh

Doctorat - Polytechnique

Doctorat - UdeM

darshan.patil@mila.quebec

Linda Peinthiere

Collaborateur·rice de recherche - Polytechnique Montreal

uphro_1@hotmail.com

mathieu.reymond@mila.quebec

Yann Pernot

Maîtrise recherche - Polytechnique

yann.pernot@mila.quebec

Mathieu Reymond

Collaborateur·rice alumni

Shaipranesh Senthilkumar

Doctorat - Polytechnique

shaipraneshgci@gmail.com

nishant.sharma@mila.quebec

Nour Shaheen

Maîtrise recherche - Polytechnique

Superviseur⋅e principal⋅e :

Amine Mhedhbi

nour.shaheen@mila.quebec

Maîtrise recherche - UdeM

Co-superviseur⋅e :

Amal Zouaq

Arjun Vaithilingam Sudhakar

Achille Sowa Samo

Doctorat - Polytechnique

achille.sowa@mila.quebec

Anabel Tan

Postdoctorat - UdeM

Co-superviseur⋅e :

Julie Hlavacek-Larrondo

anabel.tan@mila.quebec

Megh Thakkar

Maîtrise recherche - UdeM

megh.thakkar@mila.quebec

Doctorat - Polytechnique

arjun.vaithilingam-sudhakar@mila.quebec

Abdelrahman Zayed

Doctorat - Polytechnique

zayedabd@mila.quebec

Xutong Zhao

Doctorat - Polytechnique

xutong.zhao@mila.quebec

Artem Zholus

Doctorat - Polytechnique

artem.zholus@mila.quebec

Optimiser la conception CAO grâce aux LLM

Zayd Zuhri

Maîtrise recherche - UdeM

Co-superviseur⋅e :

Billets de blogue

19 décembre 2025

par

Prashant Govindarajan

Davide Baldelli

Quentin Fournier

Sarath Chandar

Lire l'article

A digital picture of Bert from Sesame street, wering black trench coat and sunglasses

3 mars 2025

NeoBERT: une nouvelle frontière pour les modèles de langage encodeurs open-source

par

Lola Le Breton

Quentin Fournier

Sarath Chandar

Lire l'article

1 octobre 2024

Comment expliquer l’IA et s’assurer que cette explication est vraie? Les modèles mesurables de fidélité vous indiquent comment y parvenir

par

Andrea Madsen

Siva Reddy

Sarath Chandar

Lire l'article

Publications

recAMPLIFY: Large-Scale Pretraining of Recursive Protein Language Models

Saurav Jha

Qianli Liao

Robert Vernon

Quentin Fournier

Protein language models (pLMs) are powerful tools for protein annotation and mutation-effect prediction, but progress has largely depended o… (voir plus)n scaling model size and data, making them expensive and prone to memorization. Test-time scaling offers a complementary path, but methods developed for autoregressive LLMs do not transfer directly: masked encoders lack a generation loop, and amino acids do not verbalize in a reasoning language. Instead, we adapt recursive pretraining to masked pLMs, where a weight-tied Transformer core refines residue-aligned hidden states over an adjustable number of cycles. We instantiate this recipe as recAMPLIFY, a pLM with 155M unique parameters trained on the same 2T tokens as AMPLIFY-350M. Despite 56% fewer parameters, it improves CASP contact recovery and three out of four supervised tasks, notably raising fold accuracy by 22.5%. Because optimal depth for recAMPLIFY varies across tasks and assays, we then introduce Gated Mixture Recurrence (GMR), a framework that decodes each protein from a probability-weighted mixture of hidden states visited during recursion. GMR employs a learned residual gate at scalar, low-rank, or per-dimension resolution to control how much of each cycle's proposed update is accepted into the state. This unifies halting and gating into a single adaptive computation objective while recovering PonderNet-style halting as a special case without gates. Layered on top of recAMPLIFY, GMR further reduces expected layer depth by up to 37%, although the accuracy trade-off varies by task. Mechanistic analyses show that residual gating controls recurrence stability beyond the trained depth and that halt depth tracks biological properties.

2026-10-18

arXiv (prépublication)

Understanding Stability of Outer Nesterov Momentum on Stochastic Pseudo-Gradients

Nishant Sharma

Laura Gomezjurado Gonzalez

Ganesh Talluri

Hiroki Naganuma

Large neural networks are commonly trained across many workers, and communication between them is a major cost. Workers therefore take sever… (voir plus)al local optimization steps before sending their net parameter displacements, or pseudo-gradients, to an outer optimizer, which often applies Nesterov momentum. We study how the outer momentum, the number of local steps and the averaging over workers control the training dynamics, and in particular their stability. For random quadratics, we derive an exact mean-square stability criterion in any dimension, which reveals that the inner batch size determines whether increasing the number of local steps eventually stabilizes or destabilizes the outer iteration. Moreover, while averaging the pseudo-gradients of independent workers can make larger outer momentum values admissible, the number of workers required for stability can grow exponentially with the number of local steps. Finally, we show that the mean-square dynamics governing stability can be probed directly at intermediate neural-network checkpoints using shared-minibatch perturbations. We test our theoretical findings across networks, and measure their consequences for training loss.

2026-10-14

arXiv (prépublication)

RoboJEPA: Scaling Robotic Latent World Models

Artem Zholus

Nicolas Beltran-Velez

Jianhao Yuan

Tushar Nagarajan

Daniel Severo

Koustuv Sinha

Michal Drozdzal

Adriana Romero Soriano

Jeannette Bohg

Nicolas Ballas

Mahmoud Assran

Latent world models have shown a remarkable ability to predict future states and to plan in the real world. In practice, however, we lack a … (voir plus)principled way to estimate how their capabilities scale with model size, data, and compute, an open problem that slows progress in the field. In this work we present RoboJEPA, a world model based on the Joint Embedding Predictive Architecture (JEPA) and trained on a large-scale dataset spanning 12 robotic embodiments. We show that RoboJEPA's imagination error, the error of its latent rollouts, follows a second-order power law in compute, allowing us to predict model quality well beyond the scale at which the law is fit. We further show that downstream robotic planning performance improves predictably with compute, and that imagination error is strongly correlated with it, making it a reliable proxy for real-robot evaluation. Finally, we demonstrate that latent world models can be deployed zero-shot as robotic agents, planning toward a single goal image to solve tasks requiring long-horizon planning on real hardware. We release all model checkpoints together with our training and robot deployment code. To our knowledge, this is the first work to establish scaling laws for multi-embodiment robotic world models trained on real robot data, and RoboJEPA, at 8B parameters, is the largest JEPA predictor model trained to date.

2026-10-06

arXiv (prépublication)

What Does Layer-Importance Reveal About Transformers and State-Space Models?

Transformers and state-space models (SSMs) are the two dominant families of sequence models, and a central open question is how far the anal… (voir plus)ytical knowledge built for transformers transfers to SSMs. We address this through the lens of layer importance which underpins compression, selective fine-tuning, and interpretability across both families. We decompose layer importance into two distinct notions. \emph{Necessity} captures how much the pretrained model depends on a layer's existing contribution, measured by the loss increase from bypassing it. \emph{Plasticity} captures where the model absorbs new information during fine-tuning, measured by the magnitude of task-specific weight updates. Our analysis reveals that the two families behave fundamentally differently: in every evaluated residual transformer up to

2026-09-29

WiML @ Neural Information Processing Systems (publié)

Sparse Koopman Autoencoders Identify Local Dynamical Regimes in Multibasin Systems

Aidan Li

Uday Kiran Reddy Tadipatri

Mahan Fathi

Ross Goroshin

Koopman autoencoders (KAEs) seek a higher-dimensional latent representation in which nonlinear dynamics evolve linearly. However, many inter… (voir plus)esting systems have multiple basins of attraction, and both theoretical and empirical work has shown these multibasin systems cannot generally admit a single finite-dimensional global Koopman embedding under standard assumptions. We posit that encoders with a sparsity-inducing objective encouraging few active latent coefficients will provide latent supports as an inspectable basin-modeling principle for Koopman autoencoders. We use these encoders producing sparse latents in training Sparse Koopman Autoencoders (SKAEs) without basin labels or other regime annotations, and treat the learned latent supports as model-produced regime variables after training. Across a range of procedurally generated multibasin systems and chaotic flows, we show that SKAEs have superior forecasting performance compared to dense-latent KAEs. We also perform a mechanistic study that shows latent supports produced by SKAEs are both essential for the quality of the representation and useful for identifying basins on held-out basin interior states, whereas dense-latent KAEs collapse to an uninformative single family. These results identify sparse latents and their corresponding supports as label-free, interpretable regime variables for Koopman learning in nonlinear systems with multiple local dynamical laws.

2026-08-28

arXiv (prépublication)

CoPeP: Benchmarking Continual Pretraining for Protein Language Models

Quentin Fournier

Protein language models (pLMs) have recently gained significant attention for their ability to uncover relationships between sequence, struc… (voir plus)ture, and function from evolutionary statistics, thereby accelerating therapeutic drug discovery. These models learn from large protein databases that are continuously updated by the biology community and whose dynamic nature motivates the application of continual learning, not only to keep up with the ever-growing data, but also as an opportunity to take advantage of the temporal meta-information that is created during this process. As a result, we introduce the Continual Pretraining of Protein Language Models (CoPeP) benchmark, a novel benchmark for evaluating continual learning approaches on pLMs. Specifically, we curate a sequence of protein datasets derived from the UniProt Knowledgebase spanning a decade and define metrics to assess pLM performance across 31 protein understanding tasks. We evaluate several methods from the continual learning literature, including replay, unlearning, and plasticity-based methods, some of which have never been applied to models and data of this scale. Our findings reveal that incorporating temporal meta-information improves perplexity by up to 7% even when compared to training on data from all tasks jointly. Moreover, even at scale, several continual learning methods outperform naive continual pretraining. The CoPeP benchmark offers an exciting opportunity to study these methods at scale in an impactful real-world application.

2026-08-24

Work-In-Progress Track @ Conference on Lifelong Learning Agents (poster)

Loss Smoothing for Stable Adaptation Under Distribution Shift

In settings such as fine-tuning and reinforcement learning, neural networks are often adapted under distribution shift. Standard adaptation … (voir plus)methods typically optimize the target objective directly, inducing an abrupt change from the source training objective. This abrupt transition can distort learned representations, including features that may still be useful for the new task. We investigate whether a more gradual transition can improve adaptation. We propose loss smoothing, a simple approach that interpolates between the source and target training objectives at the start of adaptation. This smooth transition helps to preserve useful features from the source distribution while still enabling the model to specialize to the target distribution. Across controlled supervised shifts, pretrained vision adaptation, offline-to-online and online reinforcement learning, and language model fine-tuning, we find that loss smoothing consistently improves performance, suggesting that smoother objective transitions are a broadly useful tool for model adaptation.

2026-08-24

Work-In-Progress Track @ Conference on Lifelong Learning Agents (poster)

pLM representations unlock metagenomic space beyond homology

Lola Le Breton

David Heurtel-Depeiges

Douglas C. Millar

Lara E. Zetzsche

Robert M. Vernon

Christopher J. Langmead

Quentin Fournier

Metagenomic sequencing has uncovered billions of proteins from uncultured microorganisms, vastly expanding the known protein space. Yet most… (voir plus) remain functionally inaccessible because existing annotation methods depend on close homologs or accurate structure predictions. Here, we show that protein language models (pLMs) can unlock this diversity only when their training data are appropriately curated. We introduce Residue Embedding Diversity (RED), a metric for protein quality assessment orders of magnitude cheaper than likelihood, and a calibration task that measures model alignment with natural evolutionary distributions. We discover a fundamental trade-off between evolutionary calibration and structural modeling, establishing training data composition as a primary determinant of pLM behavior. Finally, we successfully retrieve diverse enzyme candidates from billions of metagenomic sequences and validate their expression in vivo .

2026-07-29

bioRxiv (prépublication)

Addressing Sparse-Rewards in RL with Scalable Hierarchical Novel Eigen Options

Isabeau Prémont-Schwarz

Temporally extended exploration via graph Laplacian-based options is a promising approach to sparse-reward reinforcement learning (RL), but … (voir plus)existing methods either do not explicitly target novelty or fail to scale to pixel-based domains under function approximation. Novel Exploration via Orthogonality (NEO) addresses the first issue by constructing options that navigate from highly visited regions toward less visited ones, yet prior results were limited to settings where exact eigenvectors can be computed. We present a scalable extension of NEO to pixel-based domains, built on three contributions. First, we use a novelty-weighted continuous Laplacian graph-drawing objective, which enables RL with continuous observations. Second, we embed the resulting eigen-potential options within a hierarchical reinforcement learning framework, enabling coherent temporally extended behavior. Third, we observe that learned eigen-potential rewards are directional but locally unreliable under online approximation; we therefore augment each option reward with a novelty bonus, a novel design idea that proves essential for stabilizing option learning while preserving novelty-directed exploration. Together, these contributions yield stronger and more persistent exploration, enabling longer option rollouts and better access to hard-to-reach novel states. Empirically, our method significantly outperforms both the prior scalable Laplacian-option baseline and a direct extension of NEO on sparse-reward benchmarks under a fixed budget. On Montezuma's Revenge, our best variant achieves approximately 1.8x higher return than both baselines. On Venture, both baselines yield returns near zero, whereas our method achieves a return of 1135. Across seven hard ProcGen games, our method achieves approximately 3.5x and 5.6x higher aggregate normalized return than the two baselines, respectively.

2026-07-20

RLBRew @ Reinforcement Learning Conference (publié)

A systematic analysis of machine learning pipelines for robust antimicrobial resistance prediction

Alex Aselstyne

Enamundram Naga Karthik

Meriem El Azami

Romain Pogorelcnik

Quentin Fournier

Abstract Motivation Antimicrobial resistance (AMR) has been identified as a top global public health threat. Accurate AMR phenotype predicti… (voir plus)on from whole-genome sequencing data is an essential tool for accelerating clinical decision-making and mitigating resistance spread. Although many previous works have explored the use of tree-based machine learning (ML) models to predict resistance, the field lacks a systematic evaluation of the training pipeline across a variety of pathogenic species and antibiotics. Results Using nine clinically relevant species–antibiotic combinations from the NCBI antimicrobial susceptibility testing database, we present a detailed analysis of the ML pipeline and identify key factors affecting model performance and evaluation. We begin by relabelling all isolates using current CLSI minimum inhibitory concentration breakpoints to resolve inconsistencies and increase available data, resulting in up to a 19% label swap and 56% data enlargement per species– antibiotic combination. We identify several key training parameters including k -mer length, which can increase classification F1 scores by over 20 points compared to commonly used k -values, feature matrix truncation, which can induce polynomial time reductions with limited performance reduction, and ML model class. By comparing 5-fold cross-validation with evaluation on an unseen clinical dataset, we show that random cross-validation splits—often criticized as overly optimistic—can act as a strong proxy for downstream clinical performance, yielding closer F1 scores than phylogeny-aware splits in all cases. We finally present an interpretability study which shows that over 95% of k -mers used by our models are associated with identifiable genomic features. Our results highlight the importance of feature design, evaluation protocol, and biological analysis in genomic AMR prediction, and support tree-based models as a robust and interpretable method. Availability and implementation Python code is made freely available: https://github.com/chandar-lab/amr-pred

2026-07-07

bioRxiv (prépublication)

Consistent but Miscalibrated: Evaluating LLM Limitations for Risk Communication in Natural Language

Diego Cerda-Mardini

Sreenath Madathil

LLMs are increasingly deployed as post-hoc explainers of AI-generated outputs, yet it remains unclear whether they can reliably communicate … (voir plus)probabilistic information in natural language. For this role to be viable, models must produce identical verbal descriptions for identical inputs, and select descriptions that accurately reflect the magnitude of the underlying numerical quantities. We evaluate whether nine LLMs meet these requirements within a two-stage prediction pipeline, in which an upstream model has produced probabilistic outputs characterized by their likelihood and uncertainty, and LLMs are tasked with selecting an appropriate verbal descriptor for each. We simulate predictions from an upstream model by taking samples from a Beta distribution parameterized by its mode and prior sample size. We then prompt LLMs to explain these predictions under six domain contexts and with ten temperature settings, and repeating each experiment ten times. We find that LLMs are generally consistent but miscalibrated, with substantially weaker performance on uncertainty than on likelihood tasks. Providing models with precomputed summary statistics (mode and prior sample size) reduced sensitivity to contextual framing but did not resolve the underlying miscalibration, suggesting that the bottleneck resides in the verbalization step itself. These findings indicate that current LLMs do not yet constitute reliable zero-shot standalone risk communication tools for probabilistic predictions.

2026-07-03

arXiv (prépublication)

NeuroFaith: Evaluating Mechanistic Faithfulness of LLM Free Text Self-Explanation at the Concept Level

Milan Bhan

Jean-Noël Vittaut

Nicolas Chesneau

Marie-Jeanne Lesot

Large Language Models (LLMs) can generate plausible free text self-explanations to justify their answers. However, these natural language ex… (voir plus)planations may not accurately reflect the model's actual reasoning process, indicating a lack of faithfulness. Existing faithfulness evaluation methods rely primarily on behavioral tests or computational block analysis without examining the semantic content of internal neural representations. This paper proposes NeuroFaith, a flexible framework that measures the faithfulness of LLM free text self-explanation by identifying key concepts within explanations and mechanistically testing whether these concepts actually influence the model's predictions. We show the versatility of NeuroFaith across 2-hop reasoning and classification tasks. Additionally, we develop a linear faithfulness probe based on NeuroFaith to detect unfaithful self-explanations from representation space and improve faithfulness through steering. NeuroFaith provides a principled approach to evaluating and enhancing the faithfulness of LLM free text self-explanations, addressing critical needs for trustworthy AI systems.

2026-06-10

ICML.cc/2026/Workshop/Mech_Interp (poster)