Sarath Chandar

istabrak.abbes@mila.quebec

Biography

Sarath Chandar is an associate professor at Polytechnique Montreal's Department of Computer and Software Engineering, where he leads the Chandar Research Lab. He is also a Core Academic Member at Mila – Quebec Artificial Intelligence Institute and holds a Canada CIFAR AI Chair and the Canada Research Chair in Lifelong Machine Learning.

Chandar’s research interests include lifelong learning, deep learning, optimization, reinforcement learning and natural language processing. To promote research in lifelong learning, Chandar created the Conference on Lifelong Learning Agents (CoLLAs) in 2022, for which he served as program chair in 2022 and 2023.

He has a PhD from Université de Montréal and an MSc (By Research) from the Indian Institute of Technology Madras.

Current Students

Istabrak Abbes

PhD - Université de Montréal

Alex Aselstyne

Master's Research - Polytechnique Montréal

alexander.aselstyne@mila.quebec

davide.baldelli@mila.quebec

Davide Baldelli

PhD - Polytechnique Montréal

Co-supervisor :

Amal Zouaq

diego.cerda-mardini@mila.quebec

Milan Bhan

Collaborating researcher

milan.bhan@mila.quebec

Diego Cerda Mardini

Master's Research - McGill University

Prashant Govindarajan

PhD - Polytechnique Montréal

prashant.govindarajan@mila.quebec

nathanael.haas@mila.quebec

Simon Guiroy

PhD - Université de Montréal

Principal supervisor :

Research Intern - Polytechnique Montréal

Maryam Hashemzadeh

PhD - Université de Montréal

maryam.hashemzadeh@mila.quebec

david.heurtel-depeiges@mila.quebec

David Heurtel--Depeiges

PhD - Polytechnique Montréal

Jerry Huang

PhD - Université de Montréal

jerry.huang@mila.quebec

Kanishk Jain

PhD - Université de Montréal

Co-supervisor :

Ross Goroshin

kanishk.jain@mila.quebec

Saurav Jha

Postdoctorate - Polytechnique Montréal

saurav.jha@mila.quebec

Amir Ardalan Kalantari Dehaghi

PhD - Université de Montréal

Master's Research - Polytechnique Montréal

antoine.lafrance@mila.quebec

lola.lebreton@mila.quebec

Lola Le Breton

PhD - Polytechnique Montréal

Tom Lee

Postdoctorate - Polytechnique Montréal

thomas.lee@mila.quebec

ekaterina.lobacheva@mila.quebec

Aidan Li

Master's Research - Université de Montréal

Co-supervisor :

Postdoctorate - Université de Montréal

PhD - Polytechnique Montréal

mansi.maheshwari@mila.quebec

Roshan Munirathinam Sankaran Balaji

Collaborating researcher - Polytechnique Montréal

roshan.munirathinam@mila.quebec

nilaksh.nilaksh@mila.quebec

Hadi NekoeiQachkanloo

PhD - Université de Montréal

nekoeihe@mila.quebec

Nilaksh Nilaksh

PhD - Polytechnique Montréal

PhD - Université de Montréal

darshan.patil@mila.quebec

Linda Peinthiere

Collaborating researcher - Polytechnique Montréal Montreal

uphro_1@hotmail.com

mathieu.reymond@mila.quebec

Yann Pernot

Master's Research - Polytechnique Montréal

yann.pernot@mila.quebec

Mathieu Reymond

Collaborating Alumni

Shaipranesh Senthilkumar

PhD - Polytechnique Montréal

shaipraneshgci@gmail.com

nishant.sharma@mila.quebec

Nour Shaheen

Master's Research - Polytechnique Montréal

Principal supervisor :

Amine Mhedhbi

nour.shaheen@mila.quebec

Master's Research - Université de Montréal

Co-supervisor :

Amal Zouaq

Arjun Vaithilingam Sudhakar

Achille Sowa Samo

PhD - Polytechnique Montréal

achille.sowa@mila.quebec

Anabel Tan

Postdoctorate - Université de Montréal

Co-supervisor :

Julie Hlavacek-Larrondo

anabel.tan@mila.quebec

Megh Thakkar

Master's Research - Université de Montréal

megh.thakkar@mila.quebec

PhD - Polytechnique Montréal

arjun.vaithilingam-sudhakar@mila.quebec

Abdelrahman Zayed

PhD - Polytechnique Montréal

zayedabd@mila.quebec

Xutong Zhao

PhD - Polytechnique Montréal

xutong.zhao@mila.quebec

Artem Zholus

PhD - Polytechnique Montréal

artem.zholus@mila.quebec

Improving CAD Design With LLMs

Zayd Zuhri

Master's Research - Université de Montréal

Blog Posts

December 19, 2025

Prashant Govindarajan

Davide Baldelli

Quentin Fournier

Sarath Chandar

Read the article

A digital picture of Bert from Sesame street, wering black trench coat and sunglasses

March 3, 2025

NeoBERT: A New Frontier for Open-Source Encoder Language Models

Lola Le Breton

Quentin Fournier

Sarath Chandar

Read the article

October 1, 2024

How Do We Explain AI and Ensure the Explanation Is True? Faithfulness Measurable Models Tell You How

Andrea Madsen

Siva Reddy

Sarath Chandar

Read the article

Publications

Sub-Goal Distillation: A Method to Improve Small Language Agents

Maryam Hashemzadeh

Elias Stengel-Eskin

Marc-Alexandre Côté

While Large Language Models (LLMs) have demonstrated significant promise as agents in interactive tasks, their substantial computational req… (see more)uirements and restricted number of calls constrain their practical utility, especially in long-horizon interactive tasks such as decision-making or in scenarios involving continuous ongoing tasks. To address these constraints, we propose a method for transferring the performance of an LLM with billions of parameters to a much smaller language model (770M parameters). Our approach involves constructing a hierarchical agent comprising a planning module, which learns through Knowledge Distillation from an LLM to generate sub-goals, and an execution module, which learns to accomplish these sub-goals using elementary actions. In detail, we leverage an LLM to annotate an oracle path with a sequence of sub-goals towards completing a goal. Subsequently, we utilize this annotated data to fine-tune both the planning and execution modules. Importantly, neither module relies on real-time access to an LLM during inference, significantly reducing the overall cost associated with LLM interactions to a fixed cost. In ScienceWorld, a challenging and multi-task interactive text environment, our method surpasses standard imitation learning based solely on elementary actions by 16.7% (absolute). Our analysis highlights the efficiency of our approach compared to other LLM-based methods. Our code and annotated data for distillation can be found on GitHub.

2025-02-16

Proceedings of The 3rd Conference on Lifelong Learning Agents (published)

Reza Babanezhad Harikandeh

proceedings.mlr.press

Torque-Aware Momentum

Pranshu Malviya

Goncalo Mordido

Aristide Baratin

Gintare Karolina Dziugaite

Razvan Pascanu

Efficiently exploring complex loss landscapes is key to the performance of deep neural networks. While momentum-based optimizers are widely … (see more)used in state-of-the-art setups, classical momentum can still struggle with large, misaligned gradients, leading to oscillations. To address this, we propose Torque-Aware Momentum (TAM), which introduces a damping factor based on the angle between the new gradients and previous momentum, stabilizing the update direction during training. Empirical results show that TAM, which can be combined with both SGD and Adam, enhances exploration, handles distribution shifts more effectively, and improves generalization performance across various tasks, including image classification and large language model fine-tuning, when compared to classical momentum-based optimizers.

2024-12-24

arXiv (preprint)

Balancing Context Length and Mixing Times for Reinforcement Learning at Scale

Matthew Riemer

Khimya Khetarpal

Due to the recent remarkable advances in artiﬁcial intelligence, researchers have begun to consider challenging learning problems such as … (see more)learning to generalize behavior from large ofﬂine datasets or learning online in non-Markovian environments. Meanwhile, recent advances in both of these areas have increasingly relied on conditioning policies on large context lengths. A natural question is if there is a limit to the performance beneﬁts of increasing the context length if the computation needed is available. In this work, we establish a novel theoretical result that links the context length of a policy to the time needed to reliably evaluate its performance (i.e., its mixing time) in large scale partially observable reinforcement learning environments that exhibit latent sub-task structure. This analysis underscores a key tradeoff: when we extend the context length, our policy can more effectively model non-Markovian dependencies, but this comes at the cost of potentially slower policy evaluation and as a result slower downstream learning. Moreover, our empirical results highlight the relevance of this analysis when leveraging Transformer based neural networks. This perspective will become increasingly pertinent as the ﬁeld scales towards larger and more realistic environments, opening up a number of potential future directions for improving the way we design learning agents.

2024-12-10

Neural Information Processing Systems (Accept (poster))

Toward Debugging Deep Reinforcement Learning Programs with RLExplorer

Rached Bouchoucha

Ahmed Haj Yahmed

Darshan Patil

Amin Nikanjam

Foutse Khomh

Deep reinforcement learning (DRL) has shown success in diverse domains such as robotics, computer games, and recommendation systems. However… (see more), like any other software system, DRL-based software systems are susceptible to faults that pose unique challenges for debugging and diagnosing. These faults often result in unexpected behavior without explicit failures and error messages, making debugging difficult and time-consuming. Therefore, automating the monitoring and diagnosis of DRL systems is crucial to alleviate the burden on developers. In this paper, we propose RLExplorer, the first fault diagnosis approach for DRL-based software systems. RLExplorer automatically monitors training traces and runs diagnosis routines based on properties of the DRL learning dynamics to detect the occurrence of DRL-specific faults. It then logs the results of these diagnoses as warnings that cover theoretical concepts, recommended practices, and potential solutions to the identified faults. We conducted two sets of evaluations to assess RLExplorer. Our first evaluation of faulty DRL samples from Stack Overflow revealed that our approach can effectively diagnose real faults in 83% of the cases. Our second evaluation of RLExplorer with 15 DRL experts/developers showed that (1) RLExplorer could identify 3.6 times more defects than manual debugging and (2) RLExplorer is easily integrated into DRL applications.

2024-10-05

IEEE International Conference on Software Maintenance and Evolution (published)

Faithfulness Measurable Masked Language Models

Andreas Madsen

Siva Reddy

A common approach to explaining NLP models is to use importance measures that express which tokens are important for a prediction. Unfortuna… (see more)tely, such explanations are often wrong despite being persuasive. Therefore, it is essential to measure their faithfulness. One such metric is if tokens are truly important, then masking them should result in worse model performance. However, token masking introduces out-of-distribution issues, and existing solutions that address this are computationally expensive and employ proxy models. Furthermore, other metrics are very limited in scope. This work proposes an inherently faithfulness measurable model that addresses these challenges. This is achieved using a novel fine-tuning method that incorporates masking, such that masking tokens become in-distribution by design. This differs from existing approaches, which are completely model-agnostic but are inapplicable in practice. We demonstrate the generality of our approach by applying it to 16 different datasets and validate it using statistical in-distribution tests. The faithfulness is then measured with 9 different importance measures. Because masking is in-distribution, importance measures that themselves use masking become consistently more faithful. Additionally, because the model makes faithfulness cheap to measure, we can optimize explanations towards maximal faithfulness; thus, our model becomes indirectly inherently explainable.

2024-07-22

International Conference on Machine Learning (Accept (Spotlight))

proceedings.mlr.press

Lookbehind-SAM: k Steps Back, 1 Step Forward

Sharpness-aware minimization (SAM) methods have gained increasing popularity by formulating the problem of minimizing both loss value and lo… (see more)ss sharpness as a minimax objective. In this work, we increase the efficiency of the maximization and minimization parts of SAM's objective to achieve a better loss-sharpness trade-off. By taking inspiration from the Lookahead optimizer, which uses multiple descent steps ahead, we propose Lookbehind, which performs multiple ascent steps behind to enhance the maximization step of SAM and find a worst-case perturbation with higher loss. Then, to mitigate the variance in the descent step arising from the gathered gradients across the multiple ascent steps, we employ linear interpolation to refine the minimization step. Lookbehind leads to a myriad of benefits across a variety of tasks. Particularly, we show increased generalization performance, greater robustness against noisy weights, as well as improved learning and less catastrophic forgetting in lifelong learning settings. Our code is available at https://github.com/chandar-lab/Lookbehind-SAM.

2024-07-22

International Conference on Machine Learning (Accept (Poster))

Reza Babanezhad Harikandeh

proceedings.mlr.press

Promoting Exploration in Memory-Augmented Adam using Critical Momenta

Pranshu Malviya

Goncalo Mordido

Aristide Baratin

Adaptive gradient-based optimizers, notably Adam, have left their mark in training large-scale deep learning models, offering fast convergen… (see more)ce and robustness to hyperparameter settings. However, they often struggle with generalization, attributed to their tendency to converge to sharp minima in the loss landscape. To address this, we propose a new memory-augmented version of Adam that encourages exploration towards flatter minima by incorporating a buffer of critical momentum terms during training. This buffer prompts the optimizer to overshoot beyond narrow minima, promoting exploration. Through comprehensive analysis in simple settings, we illustrate the efficacy of our approach in increasing exploration and bias towards flatter minima. We empirically demonstrate that it can improve model performance for image classification on ImageNet and CIFAR10/100, language modelling on Penn Treebank, and online learning tasks on TinyImageNet and 5-dataset. Our code is available at https://github.com/chandar-lab/CMOptimizer.

2024-06-08

Transactions on Machine Learning Research (accepted)

Predicting the Impact of Model Expansion through the Minima Manifold: A Loss Landscape Perspective

The optimal model for a given task is often challenging to determine, requiring training multiple models from scratch which becomes prohibit… (see more)ive as dataset and model sizes grow. A more efficient alternative is to reuse smaller pre-trained models by expanding them, however, this is not widely adopted as how this impacts training dynamics remains poorly understood. While prior works have introduced statistics to measure these effects, they remain flawed. To rectify this, we offer a new approach for understanding and quantifying the impact of expansion through the lens of the loss landscape, which has been shown to contain a manifold of linearly connected minima. Building on this new perspective, we propose a metric to study the impact of expansion by estimating the size of the manifold. Experimental results show a clear relationship between gains in performance and manifold size, enabling the comparison of candidate models and presenting a first step towards expanding models more reliably based on geometric properties of the loss landscape.

2024-05-23

arXiv (preprint)

Interpretability Needs a New Paradigm

Andreas Madsen

Himabindu Lakkaraju

Siva Reddy

2024-05-07

arXiv (preprint)

Intelligent Switching in Reset-Free RL

Darshan Patil

Glen Berseth

In the real world, the strong episode resetting mechanisms that are needed to train agents in simulation are unavailable. The \textit{resett… (see more)ing} assumption limits the potential of reinforcement learning in the real world, as providing resets to an agent usually requires the creation of additional handcrafted mechanisms or human interventions. Recent work aims to train agents (\textit{forward}) with learned resets by constructing a second (\textit{backward}) agent that returns the forward agent to the initial state. We find that the termination and timing of the transitions between these two agents are crucial for algorithm success. With this in mind, we create a new algorithm, Reset Free RL with Intelligently Switching Controller (RISC) which intelligently switches between the two agents based on the agent's confidence in achieving its current goal. Our new method achieves state-of-the-art performance on several challenging environments for reset-free RL.

2024-05-06

International Conference on Learning Representations (Accept (poster))

Learning conditional policies for crystal design using offline reinforcement learning

Prashant Govindarajan

Santiago Miret

Jarrid Rector-Brooks

Mariano Phielipp

Conservative Q-learning for band-gap conditioned crystal design with DFT evaluations – the model is trained on trajectories constructed fr… (see more)om crystals in the Materials Project. Results indicate promising performance for lower band gap targets.

2023-12-31

Digital Discovery (published)

An Empirical Investigation of the Role of Pre-training in Lifelong Learning

Sanket Vaibhav Mehta

Darshan Patil

Emma Strubell

The lifelong learning paradigm in machine learning is an attractive alternative to the more prominent isolated learning scheme not only due … (see more)to its resemblance to biological learning but also its potential to reduce energy waste by obviating excessive model re-training. A key challenge to this paradigm is the phenomenon of catastrophic forgetting. With the increasing popularity and success of pre-trained models in machine learning, we pose the question: What role does pre-training play in lifelong learning, specifically with respect to catastrophic forgetting? We investigate existing methods in the context of large, pre-trained models and evaluate their performance on a variety of text and image classification tasks, including a large-scale study using a novel data set of 15 diverse NLP tasks. Across all settings, we observe that generic pre-training implicitly alleviates the effects of catastrophic forgetting when learning multiple tasks sequentially compared to randomly initialized models. We then further investigate why pre-training alleviates forgetting in this setting. We study this phenomenon by analyzing the loss landscape, finding that pre-trained weights appear to ease forgetting by leading to wider minima. Based on this insight, we propose jointly optimizing for current task loss and loss basin sharpness to explicitly encourage wider basins during sequential fine-tuning. We show that this optimization approach outperforms several state-of-the-art task-sequential continual learning algorithms across multiple settings, occasionally even without retaining a memory that scales in size with the number of tasks.

2023-12-11

Conference on Neural Information Processing Systems (Accept (Poster))