Sarath Chandar

istabrak.abbes@mila.quebec

Biographie

Sarath Chandar est professeur associé au départment de génie informatique et génie logiciel de Polytechnique Montréal, où il dirige le laboratoire de recherche Chandar. Il est également membre académique principal à Mila – Institut québécois d’intelligence artificielle, et titulaire d'une chaire en IA Canada-CIFAR et d'une Chaire de recherche du Canada en apprentissage machine permanent.

Ses recherches portent sur l'apprentissage tout au long de la vie, l'apprentissage profond, l'optimisation, l'apprentissage par renforcement et le traitement du langage naturel. Pour promouvoir la recherche sur l'apprentissage tout au long de la vie, Sarath Chandar a créé la Conférence sur les agents d'apprentissage tout au long de la vie (CoLLAs) en 2022 et a présidé le programme en 2022 et en 2023. Il est titulaire d'un doctorat de l'Université de Montréal et d'une maîtrise en recherche de l'Indian Institute of Technology Madras.

Étudiants actuels

Istabrak Abbes

Doctorat - UdeM

Alex Aselstyne

Maîtrise recherche - Polytechnique

alexander.aselstyne@mila.quebec

davide.baldelli@mila.quebec

Davide Baldelli

Doctorat - Polytechnique

Co-superviseur⋅e :

Amal Zouaq

quentin.fournier@mila.quebec

Milan Bhan

Collaborateur·rice de recherche

milan.bhan@mila.quebec

Quentin Fournier

Visiteur de recherche indépendant - UdeM

prashant.govindarajan@mila.quebec

Prashant Govindarajan

Doctorat - Polytechnique

nathanael.haas@mila.quebec

Simon Guiroy

Doctorat - UdeM

Superviseur⋅e principal⋅e :

Stagiaire de recherche - Polytechnique

Maryam Hashemzadeh

Doctorat - UdeM

maryam.hashemzadeh@mila.quebec

david.heurtel-depeiges@mila.quebec

David Heurtel--Depeiges

Doctorat - Polytechnique

Jerry Huang

Doctorat - UdeM

jerry.huang@mila.quebec

Kanishk Jain

Doctorat - UdeM

Co-superviseur⋅e :

Ross Goroshin

kanishk.jain@mila.quebec

Saurav Jha

Postdoctorat - Polytechnique

saurav.jha@mila.quebec

Amir Ardalan Kalantari Dehaghi

Doctorat - UdeM

Visiteur de recherche indépendant - Technical University of Munich

F.kapl@web.de

Antoine Lafrance

Maîtrise recherche - Polytechnique

antoine.lafrance@mila.quebec

lola.lebreton@mila.quebec

Lola Le Breton

Doctorat - Polytechnique

Tom Lee

Postdoctorat - Polytechnique

thomas.lee@mila.quebec

ekaterina.lobacheva@mila.quebec

Aidan Li

Maîtrise recherche - UdeM

Co-superviseur⋅e :

Postdoctorat - UdeM

Doctorat - Polytechnique

mansi.maheshwari@mila.quebec

Roshan Munirathinam Sankaran Balaji

Collaborateur·rice de recherche - Polytechnique

roshan.munirathinam@mila.quebec

nilaksh.nilaksh@mila.quebec

Hadi NekoeiQachkanloo

Doctorat - UdeM

nekoeihe@mila.quebec

Nilaksh Nilaksh

Doctorat - Polytechnique

Doctorat - UdeM

darshan.patil@mila.quebec

Linda Peinthiere

Collaborateur·rice de recherche - Polytechnique Montreal

uphro_1@hotmail.com

mathieu.reymond@mila.quebec

Yann Pernot

Maîtrise recherche - Polytechnique

yann.pernot@mila.quebec

Mathieu Reymond

Collaborateur·rice alumni

Shaipranesh Senthilkumar

Doctorat - Polytechnique

shaipraneshgci@gmail.com

nishant.sharma@mila.quebec

Nour Shaheen

Maîtrise recherche - Polytechnique

Superviseur⋅e principal⋅e :

Amine Mhedhbi

nour.shaheen@mila.quebec

Maîtrise recherche - UdeM

Co-superviseur⋅e :

Amal Zouaq

Arjun Vaithilingam Sudhakar

Achille Sowa Samo

Doctorat - Polytechnique

achille.sowa@mila.quebec

Anabel Tan

Postdoctorat - UdeM

Co-superviseur⋅e :

Julie Hlavacek-Larrondo

anabel.tan@mila.quebec

Megh Thakkar

Maîtrise recherche - UdeM

megh.thakkar@mila.quebec

Doctorat - Polytechnique

arjun.vaithilingam-sudhakar@mila.quebec

Abdelrahman Zayed

Doctorat - Polytechnique

zayedabd@mila.quebec

Xutong Zhao

Doctorat - Polytechnique

xutong.zhao@mila.quebec

Artem Zholus

Doctorat - Polytechnique

artem.zholus@mila.quebec

Optimiser la conception CAO grâce aux LLM

Zayd Zuhri

Maîtrise recherche - UdeM

Co-superviseur⋅e :

Billets de blogue

19 décembre 2025

par

Prashant Govindarajan

Davide Baldelli

Quentin Fournier

Sarath Chandar

Lire l'article

A digital picture of Bert from Sesame street, wering black trench coat and sunglasses

3 mars 2025

NeoBERT: une nouvelle frontière pour les modèles de langage encodeurs open-source

par

Lola Le Breton

Quentin Fournier

Sarath Chandar

Lire l'article

1 octobre 2024

Comment expliquer l’IA et s’assurer que cette explication est vraie? Les modèles mesurables de fidélité vous indiquent comment y parvenir

par

Andrea Madsen

Siva Reddy

Sarath Chandar

Lire l'article

Publications

Reconstruction or Semantics? What Makes a Latent Space Useful for Robotic World Models

Nilaksh

Saurav Jha

Artem Zholus

2025-12-31

Neural Information Processing Systems (Accept (poster))

Squeezing More from the Stream : Learning Representation Online for Streaming Reinforcement Learning

Nilaksh

In streaming Reinforcement Learning (RL), transitions are observed and discarded immediately after a single update. While this minimizes res… (voir plus)ource usage for on-device applications, it makes agents notoriously sample-inefficient, since value-based losses alone struggle to extract meaningful representations from transient data. We propose extending Self-Predictive Representations (SPR) to the streaming pipeline to maximize the utility of every observed frame. However, due to the highly correlated samples induced by the streaming regime, naively applying this auxiliary loss results in training instabilities. Thus, we introduce orthogonal gradient updates relative to the momentum target and resolve gradient conflicts arising from streaming-specific optimizers. Validated across the Atari, MinAtar, and Octax suites, our approach systematically outperforms existing streaming baselines. Latent-space analysis, including t-SNE visualizations and effective-rank measurements, confirms that our method learns significantly richer representations, bridging the performance gap caused by the absence of a replay buffer, while remaining efficient enough to train on just a few CPU cores.

2025-12-31

International Conference on Machine Learning (Accept (regular))

The Expressive Limits of Diagonal SSMs for State-Tracking

State-Space Models (SSMs) have recently been shown to achieve strong empirical performance on a variety of long-range sequence modeling task… (voir plus)s while remaining efficient and highly-parallelizable. However, the theoretical understanding of their expressive power remains limited. In this work, we study the expressivity of input-Dependent Complex-valued Diagonal (DCD) State-Space Models (SSMs) on sequential state-tracking tasks for abstract groups. It is easy to show that a single DCD SSM layer with a universal decoder can track any Abelian group at finite precision by decomposing it into a product of cyclic groups. We show that this is tight by proving that such a model cannot track any non-Abelian group at finite precision. We further establish the expressivity of multi-layer DCD SSMs. We show that a

2025-12-31

International Conference on Learning Representations (Accept (Poster))

The Markovian Thinker

Milad Aghajohari

Kamran Chitsaz

Amirhossein Kazemnejad

Reasoning LLMs suffer from quadratic compute growth as their context length increases, making reinforcement learning with verifiable rewards… (voir plus) (RLVR) and test-time scaling prohibitively expensive. Prior work has tried to lighten the computational burden by shortening reasoning traces through pruning, summarization, or multi-stage training, but these methods remain bound to quadratic costs. We introduce Delethink, a thinking algorithm that realizes the Markovian Thinking Paradigm. Instead of producing one long monolithic reasoning trace, Delethink thinks in a sequence of chunks, the Delethink trace. Each chunk continues reasoning by referring only to a fixed number of prior tokens, which functions as a Markovian state sufficient for progressing reasoning, while deleting the rest. This preserves continuity without carrying the quadratic baggage. As a result, compute scales linearly and peak memory remains constant. In experiments, we show that Delethink can be applied directly to off-the-shelf reasoning models ranging from

2025-12-31

International Conference on Learning Representations (Accept (Poster))

Arjun Vaithilingam Sudhakar

A Generalist Hanabi Agent

Hadi Nekoei

Mathieu Reymond

Miao Liu

Janarthanan Rajendran

Traditional multi-agent reinforcement learning (MARL) systems can develop cooperative strategies through repeated interactions. However, the… (voir plus)se systems are unable to perform well on any other setting than the one they have been trained on, and struggle to successfully cooperate with unfamiliar collaborators. This is particularly visible in the Hanabi benchmark, a popular 2-to-5 player cooperative card-game which requires complex reasoning and precise assistance to other agents. Current MARL agents for Hanabi can only learn one specific game-setting (e.g., 2-player games), and play with the same algorithmic agents. This is in stark contrast to humans, who can quickly adjust their strategies to work with unfamiliar partners or situations. In this paper, we introduce Recurrent Replay Relevance Distributed DQN (R3D2), a generalist agent for Hanabi, designed to overcome these limitations. We reformulate the task using text, as language has been shown to improve transfer. We then propose a distributed MARL algorithm that copes with the resulting dynamic observation- and action-space. In doing so, our agent is the first that can play all game settings concurrently, and extend strategies learned from one setting to other ones. As a consequence, our agent also demonstrates the ability to collaborate with different algorithmic agents -- agents that are themselves unable to do so. The implementation code is available at:

2025-04-22

International Conference on Learning Representations (Accept (Poster))

Sub-Goal Distillation: A Method to Improve Small Language Agents

Maryam Hashemzadeh

Elias Stengel-Eskin

Marc-Alexandre Côté

While Large Language Models (LLMs) have demonstrated significant promise as agents in interactive tasks, their substantial computational req… (voir plus)uirements and restricted number of calls constrain their practical utility, especially in long-horizon interactive tasks such as decision-making or in scenarios involving continuous ongoing tasks. To address these constraints, we propose a method for transferring the performance of an LLM with billions of parameters to a much smaller language model (770M parameters). Our approach involves constructing a hierarchical agent comprising a planning module, which learns through Knowledge Distillation from an LLM to generate sub-goals, and an execution module, which learns to accomplish these sub-goals using elementary actions. In detail, we leverage an LLM to annotate an oracle path with a sequence of sub-goals towards completing a goal. Subsequently, we utilize this annotated data to fine-tune both the planning and execution modules. Importantly, neither module relies on real-time access to an LLM during inference, significantly reducing the overall cost associated with LLM interactions to a fixed cost. In ScienceWorld, a challenging and multi-task interactive text environment, our method surpasses standard imitation learning based solely on elementary actions by 16.7% (absolute). Our analysis highlights the efficiency of our approach compared to other LLM-based methods. Our code and annotated data for distillation can be found on GitHub.

2025-02-16

Proceedings of The 3rd Conference on Lifelong Learning Agents (publié)

Reza Babanezhad Harikandeh

Torque-Aware Momentum

Pranshu Malviya

Goncalo Mordido

Aristide Baratin

Gintare Karolina Dziugaite

Razvan Pascanu

Efficiently exploring complex loss landscapes is key to the performance of deep neural networks. While momentum-based optimizers are widely … (voir plus)used in state-of-the-art setups, classical momentum can still struggle with large, misaligned gradients, leading to oscillations. To address this, we propose Torque-Aware Momentum (TAM), which introduces a damping factor based on the angle between the new gradients and previous momentum, stabilizing the update direction during training. Empirical results show that TAM, which can be combined with both SGD and Adam, enhances exploration, handles distribution shifts more effectively, and improves generalization performance across various tasks, including image classification and large language model fine-tuning, when compared to classical momentum-based optimizers.

2024-12-24

arXiv (prépublication)

arxiv.org

Balancing Context Length and Mixing Times for Reinforcement Learning at Scale

Matthew Riemer

Khimya Khetarpal

Janarthanan Rajendran

Due to the recent remarkable advances in artiﬁcial intelligence, researchers have begun to consider challenging learning problems such as … (voir plus)learning to generalize behavior from large ofﬂine datasets or learning online in non-Markovian environments. Meanwhile, recent advances in both of these areas have increasingly relied on conditioning policies on large context lengths. A natural question is if there is a limit to the performance beneﬁts of increasing the context length if the computation needed is available. In this work, we establish a novel theoretical result that links the context length of a policy to the time needed to reliably evaluate its performance (i.e., its mixing time) in large scale partially observable reinforcement learning environments that exhibit latent sub-task structure. This analysis underscores a key tradeoff: when we extend the context length, our policy can more effectively model non-Markovian dependencies, but this comes at the cost of potentially slower policy evaluation and as a result slower downstream learning. Moreover, our empirical results highlight the relevance of this analysis when leveraging Transformer based neural networks. This perspective will become increasingly pertinent as the ﬁeld scales towards larger and more realistic environments, opening up a number of potential future directions for improving the way we design learning agents.

2024-12-10

Neural Information Processing Systems (Accept (poster))

Toward Debugging Deep Reinforcement Learning Programs with RLExplorer

Rached Bouchoucha

Ahmed Haj Yahmed

Darshan Patil

Janarthanan Rajendran

Amin Nikanjam

Foutse Khomh

Deep reinforcement learning (DRL) has shown success in diverse domains such as robotics, computer games, and recommendation systems. However… (voir plus), like any other software system, DRL-based software systems are susceptible to faults that pose unique challenges for debugging and diagnosing. These faults often result in unexpected behavior without explicit failures and error messages, making debugging difficult and time-consuming. Therefore, automating the monitoring and diagnosis of DRL systems is crucial to alleviate the burden on developers. In this paper, we propose RLExplorer, the first fault diagnosis approach for DRL-based software systems. RLExplorer automatically monitors training traces and runs diagnosis routines based on properties of the DRL learning dynamics to detect the occurrence of DRL-specific faults. It then logs the results of these diagnoses as warnings that cover theoretical concepts, recommended practices, and potential solutions to the identified faults. We conducted two sets of evaluations to assess RLExplorer. Our first evaluation of faulty DRL samples from Stack Overflow revealed that our approach can effectively diagnose real faults in 83% of the cases. Our second evaluation of RLExplorer with 15 DRL experts/developers showed that (1) RLExplorer could identify 3.6 times more defects than manual debugging and (2) RLExplorer is easily integrated into DRL applications.

2024-10-05

IEEE International Conference on Software Maintenance and Evolution (publié)

arxiv.org

Faithfulness Measurable Masked Language Models

Andreas Madsen

Siva Reddy

A common approach to explaining NLP models is to use importance measures that express which tokens are important for a prediction. Unfortuna… (voir plus)tely, such explanations are often wrong despite being persuasive. Therefore, it is essential to measure their faithfulness. One such metric is if tokens are truly important, then masking them should result in worse model performance. However, token masking introduces out-of-distribution issues, and existing solutions that address this are computationally expensive and employ proxy models. Furthermore, other metrics are very limited in scope. This work proposes an inherently faithfulness measurable model that addresses these challenges. This is achieved using a novel fine-tuning method that incorporates masking, such that masking tokens become in-distribution by design. This differs from existing approaches, which are completely model-agnostic but are inapplicable in practice. We demonstrate the generality of our approach by applying it to 16 different datasets and validate it using statistical in-distribution tests. The faithfulness is then measured with 9 different importance measures. Because masking is in-distribution, importance measures that themselves use masking become consistently more faithful. Additionally, because the model makes faithfulness cheap to measure, we can optimize explanations towards maximal faithfulness; thus, our model becomes indirectly inherently explainable.

2024-07-22

International Conference on Machine Learning (Accept (Spotlight))

Lookbehind-SAM: k Steps Back, 1 Step Forward

Sharpness-aware minimization (SAM) methods have gained increasing popularity by formulating the problem of minimizing both loss value and lo… (voir plus)ss sharpness as a minimax objective. In this work, we increase the efficiency of the maximization and minimization parts of SAM's objective to achieve a better loss-sharpness trade-off. By taking inspiration from the Lookahead optimizer, which uses multiple descent steps ahead, we propose Lookbehind, which performs multiple ascent steps behind to enhance the maximization step of SAM and find a worst-case perturbation with higher loss. Then, to mitigate the variance in the descent step arising from the gathered gradients across the multiple ascent steps, we employ linear interpolation to refine the minimization step. Lookbehind leads to a myriad of benefits across a variety of tasks. Particularly, we show increased generalization performance, greater robustness against noisy weights, as well as improved learning and less catastrophic forgetting in lifelong learning settings. Our code is available at https://github.com/chandar-lab/Lookbehind-SAM.

2024-07-22

International Conference on Machine Learning (Accept (Poster))

Reza Babanezhad Harikandeh

Promoting Exploration in Memory-Augmented Adam using Critical Momenta

Pranshu Malviya

Goncalo Mordido

Aristide Baratin

Adaptive gradient-based optimizers, notably Adam, have left their mark in training large-scale deep learning models, offering fast convergen… (voir plus)ce and robustness to hyperparameter settings. However, they often struggle with generalization, attributed to their tendency to converge to sharp minima in the loss landscape. To address this, we propose a new memory-augmented version of Adam that encourages exploration towards flatter minima by incorporating a buffer of critical momentum terms during training. This buffer prompts the optimizer to overshoot beyond narrow minima, promoting exploration. Through comprehensive analysis in simple settings, we illustrate the efficacy of our approach in increasing exploration and bias towards flatter minima. We empirically demonstrate that it can improve model performance for image classification on ImageNet and CIFAR10/100, language modelling on Penn Treebank, and online learning tasks on TinyImageNet and 5-dataset. Our code is available at https://github.com/chandar-lab/CMOptimizer.

2024-06-08

Transactions on Machine Learning Research (accepté)