Sarath Chandar

istabrak.abbes@mila.quebec

Biography

Sarath Chandar is an associate professor at Polytechnique Montreal's Department of Computer and Software Engineering, where he leads the Chandar Research Lab. He is also a Core Academic Member at Mila – Quebec Artificial Intelligence Institute and holds a Canada CIFAR AI Chair and the Canada Research Chair in Lifelong Machine Learning.

Chandar’s research interests include lifelong learning, deep learning, optimization, reinforcement learning and natural language processing. To promote research in lifelong learning, Chandar created the Conference on Lifelong Learning Agents (CoLLAs) in 2022, for which he served as program chair in 2022 and 2023.

He has a PhD from Université de Montréal and an MSc (By Research) from the Indian Institute of Technology Madras.

Current Students

Istabrak Abbes

PhD - Université de Montréal

Alex Aselstyne

Master's Research - Polytechnique Montréal

alexander.aselstyne@mila.quebec

davide.baldelli@mila.quebec

Davide Baldelli

PhD - Polytechnique Montréal

Co-supervisor :

Amal Zouaq

diego.cerda-mardini@mila.quebec

Milan Bhan

Collaborating researcher

milan.bhan@mila.quebec

Diego Cerda Mardini

Master's Research - McGill University

Naga Karthik Enamundram

PhD - Polytechnique Montréal

Principal supervisor :

Julien Cohen-Adad

emvnagakarthik@gmail.com

prashant.govindarajan@mila.quebec

Prashant Govindarajan

PhD - Polytechnique Montréal

maryam.hashemzadeh@mila.quebec

Simon Guiroy

PhD - Université de Montréal

Principal supervisor :

PhD - Université de Montréal

david.heurtel-depeiges@mila.quebec

David Heurtel--Depeiges

PhD - Polytechnique Montréal

Jerry Huang

PhD - Université de Montréal

jerry.huang@mila.quebec

Kanishk Jain

PhD - Université de Montréal

Co-supervisor :

Ross Goroshin

kanishk.jain@mila.quebec

Saurav Jha

Postdoctorate - Polytechnique Montréal

saurav.jha@mila.quebec

Amir Ardalan Kalantari Dehaghi

PhD - Université de Montréal

PhD - Polytechnique Montréal

lola.lebreton@mila.quebec

ekaterina.lobacheva@mila.quebec

Aidan Li

Master's Research - Université de Montréal

Co-supervisor :

Postdoctorate - Université de Montréal

PhD - Polytechnique Montréal

pranshu.malviya@mila.quebec

Roshan Munirathinam Sankaran Balaji

Collaborating researcher - Polytechnique Montréal

roshan.munirathinam@mila.quebec

nilaksh.nilaksh@mila.quebec

Hadi NekoeiQachkanloo

PhD - Université de Montréal

nekoeihe@mila.quebec

Nilaksh Nilaksh

PhD - Polytechnique Montréal

PhD - Université de Montréal

darshan.patil@mila.quebec

Linda Peinthiere

Collaborating researcher - Polytechnique Montréal Montreal

uphro_1@hotmail.com

mathieu.reymond@mila.quebec

Yann Pernot

Master's Research - Polytechnique Montréal

yann.pernot@mila.quebec

Mathieu Reymond

Collaborating Alumni

Shaipranesh Senthilkumar

PhD - Polytechnique Montréal

shaipraneshgci@gmail.com

Arjun Vaithilingam Sudhakar

Nour Shaheen

Master's Research - Polytechnique Montréal

Principal supervisor :

Amine Mhedhbi

nour.shaheen@mila.quebec

PhD - Polytechnique Montréal

achille.sowa@mila.quebec

Anabel Tan

Postdoctorate - Université de Montréal

anabel.tan@mila.quebec

Megh Thakkar

Master's Research - Université de Montréal

megh.thakkar@mila.quebec

PhD - Polytechnique Montréal

arjun.vaithilingam-sudhakar@mila.quebec

Abdelrahman Zayed

PhD - Polytechnique Montréal

zayedabd@mila.quebec

Xutong Zhao

PhD - Polytechnique Montréal

xutong.zhao@mila.quebec

Artem Zholus

PhD - Polytechnique Montréal

artem.zholus@mila.quebec

Improving CAD Design With LLMs

Blog Posts

December 19, 2025

Prashant Govindarajan

Davide Baldelli

Quentin Fournier

Sarath Chandar

Read the article

A digital picture of Bert from Sesame street, wering black trench coat and sunglasses

March 3, 2025

NeoBERT: A New Frontier for Open-Source Encoder Language Models

Lola Le Breton

Quentin Fournier

Sarath Chandar

Read the article

October 1, 2024

How Do We Explain AI and Ensure the Explanation Is True? Faithfulness Measurable Models Tell You How

Andrea Madsen

Siva Reddy

Sarath Chandar

Read the article

Publications

A Brief Study on the Effects of Training Generative Dialogue Models with a Semantic loss

Prasanna Parthasarathi

Mohamed Abdelsalam

Joelle Pineau

Neural models trained for next utterance generation in dialogue task learn to mimic the n-gram sequences in the training set with training o… (see more)bjectives like negative log-likelihood (NLL) or cross-entropy. Such commonly used training objectives do not foster generating alternate responses to a context. But, the effects of minimizing an alternate training objective that fosters a model to generate alternate response and score it on semantic similarity has not been well studied. We hypothesize that a language generation model can improve on its diversity by learning to generate alternate text during training and minimizing a semantic loss as an auxiliary objective. We explore this idea on two different sized data sets on the task of next utterance generation in goal oriented dialogues. We make two observations (1) minimizing a semantic objective improved diversity in responses in the smaller data set (Frames) but only as-good-as minimizing the NLL in the larger data set (MultiWoZ) (2) large language model embeddings can be more useful as a semantic loss objective than as initialization for token embeddings.

2021-06-30

Proceedings of the 22nd Annual Meeting of the Special Interest Group on Discourse and Dialogue (published)

Continuous Coordination As a Realistic Scenario for Lifelong Learning

Hadi Nekoei

Akilesh Badrinaaraayanan

Aaron Courville

Current deep reinforcement learning (RL) algorithms are still highly task-specific and lack the ability to generalize to new environments. L… (see more)ifelong learning (LLL), however, aims at solving multiple tasks sequentially by efficiently transferring and using knowledge between tasks. Despite a surge of interest in lifelong RL in recent years, the lack of a realistic testbed makes robust evaluation of LLL algorithms difficult. Multi-agent RL (MARL), on the other hand, can be seen as a natural scenario for lifelong RL due to its inherent non-stationarity, since the agents' policies change over time. In this work, we introduce a multi-agent lifelong learning testbed that supports both zero-shot and few-shot settings. Our setup is based on Hanabi -- a partially-observable, fully cooperative multi-agent game that has been shown to be challenging for zero-shot coordination. Its large strategy space makes it a desirable environment for lifelong RL tasks. We evaluate several recent MARL methods, and benchmark state-of-the-art LLL algorithms in limited memory and computation regimes to shed light on their strengths and weaknesses. This continual learning paradigm also provides us with a pragmatic way of going beyond centralized training which is the most commonly used training protocol in MARL. We empirically show that the agents trained in our setup are able to coordinate well with unseen agents, without any additional assumptions made by previous works. The code and all pre-trained models are available at https://github.com/chandar-lab/Lifelong-Hanabi.

2021-06-30

Proceedings of the 38th International Conference on Machine Learning (published)

proceedings.mlr.press

Towered Actor Critic For Handling Multiple Action Types In Reinforcement Learning For Drug Discovery

Sai Krishna Gottipati

Yashaswi Pathak

Boris Sattarov

Sahir

Rohan Nuttall

Mohammad Amini

Matthew E. Taylor

Reinforcement learning (RL) has made significant progress in both abstract and real-world domains, but the majority of state-of-the-art algo… (see more)rithms deal only with monotonic actions. However, some applications require agents to reason over different types of actions. Our application simulates reaction-based molecule generation, used as part of the drug discovery pipeline, and includes both uni-molecular and bi-molecular reactions. This paper introduces a novel framework, towered actor critic (TAC), to handle multiple action types. The TAC framework is general in that it is designed to be combined with any existing RL algorithms for continuous action space. We combine it with TD3 to empirically obtain significantly better results than existing methods in the drug discovery setting. TAC is also applied to RL benchmarks in OpenAI Gym and results show that our framework can improve, or at least does not hurt, performance relative to standard TD3.

2021-05-17

AAAI Conference on Artificial Intelligence (published)

Learning to Navigate the Synthetically Accessible Chemical Space Using Reinforcement Learning

Sai Krishna Gottipati

Boris Sattarov

Sufeng Niu

Yashaswi Pathak

Haoran Wei

Karam J. Thomas

Connor W. Coley

Over the last decade, there has been significant progress in the field of machine learning for de novo drug design, particularly in deep gen… (see more)erative models. However, current generative approaches exhibit a significant challenge as they do not ensure that the proposed molecular structures can be feasibly synthesized nor do they provide the synthesis routes of the proposed small molecules, thereby seriously limiting their practical applicability. In this work, we propose a novel forward synthesis framework powered by reinforcement learning (RL) for de novo drug design, Policy Gradient for Forward Synthesis (PGFS), that addresses this challenge by embedding the concept of synthetic accessibility directly into the de novo drug design system. In this setup, the agent learns to navigate through the immense synthetically accessible chemical space by subjecting commercially available small molecule building blocks to valid chemical reactions at every time step of the iterative virtual multi-step synthesis process. The proposed environment for drug discovery provides a highly challenging test-bed for RL algorithms owing to the large state space and high-dimensional continuous action space with hierarchical actions. PGFS achieves state-of-the-art performance in generating structures with high QED and penalized clogP. Moreover, we validate PGFS in an in-silico proof-of-concept associated with three HIV targets. Finally, we describe how the end-to-end training conceptualized in this study represents an important paradigm in radically expanding the synthesizable chemical space and automating the drug discovery process.

2020-07-13

International Conference on Machine Learning (Accept)

proceedings.mlr.press

Toward Training Recurrent Neural Networks for Lifelong Learning.

Shagun Sodhani

Yoshua Bengio

Catastrophic forgetting and capacity saturation are the central challenges of any parametric lifelong learning system. In this work, we stud… (see more)y these challenges in the context of sequential supervised learning with an emphasis on recurrent neural networks. To evaluate the models in the lifelong learning setting, we propose a curriculum-based, simple, and intuitive benchmark where the models are trained on tasks with increasing levels of difficulty. To measure the impact of catastrophic forgetting, the model is tested on all the previous tasks as it completes any task. As a step toward developing true lifelong learning systems, we unify gradient episodic memory (a catastrophic forgetting alleviation approach) and Net2Net (a capacity expansion approach). Both models are proposed in the context of feedforward networks, and we evaluate the feasibility of using them for recurrent networks. Evaluation on the proposed benchmark shows that the unified model is more suitable than the constituent models for lifelong learning setting.

2019-12-31

Neural Computation (published)

Towards Non-Saturating Recurrent Units for Modelling Long-Term Dependencies

Chinnadhurai Sankar

Eugene Vorontsov

Samira Ebrahimi Kahou

Yoshua Bengio

Modelling long-term dependencies is a challenge for recurrent neural networks. This is primarily due to the fact that gradients vanish durin… (see more)g training, as the sequence length increases. Gradients can be attenuated by transition operators and are attenuated or dropped by activation functions. Canonical architectures like LSTM alleviate this issue by skipping information through a memory mechanism. We propose a new recurrent architecture (Non-saturating Recurrent Unit; NRU) that relies on a memory mechanism but forgoes both saturating activation functions and saturating gates, in order to further alleviate vanishing gradients. In a series of synthetic and real world tasks, we demonstrate that the proposed model is the only model that performs among the top 2 models across all tasks with and without long-term dependencies, when compared against a range of other architectures.

2019-07-16

AAAI Conference on Artificial Intelligence (published)

Do Neural Dialog Systems Use the Conversation History Effectively? An Empirical Study

Christopher Pal

Neural generative models have been become increasingly popular when building conversational agents. They offer flexibility, can be easily ad… (see more)apted to new domains, and require minimal domain engineering. A common criticism of these systems is that they seldom understand or use the available dialog history effectively. In this paper, we take an empirical approach to understanding how these models use the available dialog history by studying the sensitivity of the models to artificially introduced unnatural changes or perturbations to their context at test time. We experiment with 10 different types of perturbations on 4 multi-turn dialog datasets and find that commonly used neural dialog architectures like recurrent and transformer-based seq2seq models are rarely sensitive to most perturbations such as missing or reordering utterances, shuffling words, etc. Also, by open-sourcing our code, we believe that it will serve as a useful diagnostic tool for evaluating dialog systems in the future.

2018-12-31

Association for Computational Linguistics (published)

Environments for Lifelong Reinforcement Learning

To achieve general artificial intelligence, reinforcement learning (RL) agents should learn not only to optimize returns for one specific ta… (see more)sk but also to constantly build more complex skills and scaffold their knowledge about the world, without forgetting what has already been learned. In this paper, we discuss the desired characteristics of environments that can support the training and evaluation of lifelong reinforcement learning agents, review existing environments from this perspective, and propose recommendations for devising suitable environments in the future.

2018-11-25

ArXiv (preprint)

Dynamic Neural Turing Machine with Continuous and Discrete Addressing Schemes

We extend the neural Turing machine (NTM) model into a dynamic neural Turing machine (D-NTM) by introducing trainable address vectors. This … (see more)addressing scheme maintains for each memory cell two separate vectors, content and address vectors. This allows the D-NTM to learn a wide variety of location-based addressing strategies, including both linear and nonlinear ones. We implement the D-NTM with both continuous and discrete read and write mechanisms. We investigate the mechanisms and effects of learning to read and write into a memory through experiments on Facebook bAbI tasks using both a feedforward and GRU controller. We provide extensive analysis of our model and compare different variations of neural Turing machines on this task. We show that our model outperforms long short-term memory and NTM variants. We provide further experimental results on the sequential [Formula: see text]MNIST, Stanford Natural Language Inference, associative recall, and copy tasks.

2018-03-31

Neural Computation (published)

openreview.net

A Deep Reinforcement Learning Chatbot (Short Version)

Iulian V. Serban

Mathieu Germain

Michael Pieper

Nan Rosemary Ke

Sai Rajeswar

Alexandre De Brébisson

Jose M. R. Sotelo

We present MILABOT: a deep reinforcement learning chatbot developed by the Montreal Institute for Learning Algorithms (MILA) for the Amazon … (see more)Alexa Prize competition. MILABOT is capable of conversing with humans on popular small talk topics through both speech and text. The system consists of an ensemble of natural language generation and retrieval models, including neural network and template-based models. By applying reinforcement learning to crowdsourced data and real-world user interactions, the system has been trained to select an appropriate response from the models in its ensemble. The system has been evaluated through A/B testing with real-world users, where it performed significantly better than other systems. The results highlight the potential of coupling ensemble systems with deep reinforcement learning as a fruitful path for developing real-world, open-domain conversational agents.

2017-12-31

arXiv (preprint)

A Deep Reinforcement Learning Chatbot

Iulian V. Serban

Mathieu Germain

Michael Pieper

Nan Rosemary Ke

Sai Mudumba

Alexandre De Brébisson

Jose M. R. Sotelo

We present MILABOT: a deep reinforcement learning chatbot developed by the Montreal Institute for Learning Algorithms (MILA) for the Amazon … (see more)Alexa Prize competition. MILABOT is capable of conversing with humans on popular small talk topics through both speech and text. The system consists of an ensemble of natural language generation and retrieval models, including template-based models, bag-of-words models, sequence-to-sequence neural network and latent variable neural network models. By applying reinforcement learning to crowdsourced data and real-world user interactions, the system has been trained to select an appropriate response from the models in its ensemble. The system has been evaluated through A/B testing with real-world users, where it performed significantly better than many competing systems. Due to its machine learning architecture, the system is likely to improve with additional data.

2017-09-06

ArXiv (preprint)