Sarath Chandar Anbil Parthipan

ali-rahimi.kalahroudi@mila.quebec

Biographie

Sarath Chandar est professeur adjoint à Polytechnique Montréal, où il dirige le laboratoire de recherche Chandar. Il est également membre du corps professoral de Mila – Institut québécois d’intelligence artificielle, et titulaire d'une chaire en IA Canada-CIFAR et d'une Chaire de recherche du Canada en apprentissage machine permanent. Ses recherches portent sur l'apprentissage tout au long de la vie, l'apprentissage profond, l'optimisation, l'apprentissage par renforcement et le traitement du langage naturel. Pour promouvoir la recherche sur l'apprentissage tout au long de la vie, Sarath Chandar a créé la Conférence sur les agents d'apprentissage tout au long de la vie (CoLLAs) en 2022 et a présidé le programme en 2022 et en 2023. Il est titulaire d'un doctorat de l'Université de Montréal et d'une maîtrise en recherche de l'Indian Institute of Technology Madras.

Étudiants actuels

Abdelrahman Zayed

Doctorat - Polytechnique Montréal

zayedabd@mila.quebec

Ali Rahimi-Kalahroudi

Maîtrise recherche - Université de Montréal

Amir Ardalan Kalantari Dehaghi

Collaborateur·rice alumni

Doctorat - Polytechnique Montréal

Co-superviseur⋅e :

Siva Reddy

andreas.madsen@mila.quebec

Maîtrise recherche - Polytechnique Montréal

antoine.clavaud@mila.quebec

Arjun Vaithilingam Sudhakar

Doctorat - Polytechnique Montréal

arjun.vaithilingam-sudhakar@mila.quebec

Artem Zholus

Doctorat - Polytechnique Montréal

artem.zholus@mila.quebec

Doctorat - Université de Montréal

darshan.patil@mila.quebec

francois.rivest@mila.quebec

Francois Rivest

Visiteur de recherche indépendant

goncalo-filipe.torcato-mordido@mila.quebec

Gabriele Prato

Doctorat - Université de Montréal

Postdoctorat - Polytechnique Montréal

Hadi NekoeiQachkanloo

Doctorat - Université de Montréal

nekoeihe@mila.quebec

Ista Abbes

Maîtrise recherche - Université de Montréal

istabrak.abbes@mila.quebec

Janarthanan Rajendran

Postdoctorat - Université de Montréal

Co-superviseur⋅e :

Doina Precup

janarthanan.rajendran@mila.quebec

jarrid.rector-brooks@mila.quebec

Jarrid Rector-Brooks

Doctorat - Université de Montréal

Superviseur⋅e principal⋅e :

Yoshua Bengio

Jerry Huang

Maîtrise recherche - Université de Montréal

jerry.huang@mila.quebec

Kshitij Gupta

Collaborateur·rice alumni - Université de Montréal

Superviseur⋅e principal⋅e :

Irina Rish

kshitij.gupta@mila.quebec

Lola Le Breton

Maîtrise recherche - Polytechnique Montréal

lola.lebreton@mila.quebec

maryam.hashemzadeh@mila.quebec

Louis Clouatre

Doctorat - Polytechnique Montréal

Superviseur⋅e principal⋅e :

Amal Zouaq

clouatrl@mila.quebec

Maryam Hashemzadeh

Doctorat - None

Mathieu Duchesneau

Doctorat - Université de Montréal

Postdoctorat

mathieu.reymond@mila.quebec

maziar.sargordi@mila.quebec

Maziar Sargordi

Doctorat - Polytechnique Montréal

Co-superviseur⋅e :

Amal Zouaq

mohammad-reza.samsami@mila.quebec

Megh Thakkar

Maîtrise recherche - Université de Montréal

megh.thakkar@mila.quebec

Mohammad R. Samsami

Maîtrise recherche - Université de Montréal

naga-karthik.enamundram@mila.quebec

Naga Karthik Enamundram

Doctorat - Polytechnique Montréal

Superviseur⋅e principal⋅e :

Julien Cohen-Adad

Doctorat - Polytechnique Montréal

pranshu.malviya@mila.quebec

prashant.govindarajan@mila.quebec

Prashant Govindarajan

Doctorat - Polytechnique Montréal

Simon Guiroy

Doctorat - Université de Montréal

Superviseur⋅e principal⋅e :

Xutong Zhao

Doctorat - Polytechnique Montréal

xutong.zhao@mila.quebec

Publications

Promoting Exploration in Memory-Augmented Adam using Critical Momenta

Pranshu Malviya

Goncalo Mordido

Aristide Baratin

Reza Babanezhad Harikandeh

Jerry Huang

Simon Lacoste-Julien

Razvan Pascanu

Adaptive gradient-based optimizers, particularly Adam, have left their mark in training large-scale deep learning models. The strength of su… (voir plus)ch optimizers is that they exhibit fast convergence while being more robust to hyperparameter choice. However, they often generalize worse than non-adaptive methods. Recent studies have tied this performance gap to flat minima selection: adaptive methods tend to find solutions in sharper basins of the loss landscape, which in turn hurts generalization. To overcome this issue, we propose a new memory-augmented version of Adam that promotes exploration towards flatter minima by using a buffer of critical momentum terms during training. Intuitively, the use of the buffer makes the optimizer overshoot outside the basin of attraction if it is not wide enough. We empirically show that our method improves the performance of several variants of Adam on standard supervised language modelling and image classification tasks.

2024-06-09

TMLR (accepté)

On the Costs and Benefits of Adopting Lifelong Learning for Software Analytics -- Empirical Study on Brown Build and Risk Prediction

Doriane Olewicki

Sarra Habchi

Mathieu Nayrolles

Mojtaba Faramarzi

Bram Adams

Nowadays, software analytics tools using machine learning (ML) models to, for example, predict the risk of a code change are well establishe… (voir plus)d. However, as the goals of a project shift over time, and developers and their habits change, the performance of said models tends to degrade (drift) over time. Current retraining practices typically require retraining a new model from scratch on a large updated dataset when performance decay is observed, thus incurring a computational cost; also there is no continuity between the models as the past model is discarded and ignored during the new model training. Even though the literature has taken interest in online learning approaches, those have rarely been integrated and evaluated in industrial environments. This paper evaluates the use of lifelong learning (LL) for industrial use cases at Ubisoft, evaluating both the performance and the required computational effort in comparison to the retraining-from-scratch approaches commonly used by the industry. LL is used to continuously build and maintain ML-based software analytics tools using an incremental learner that progressively updates the old model using new data. To avoid so-called"catastrophic forgetting"of important older data points, we adopt a replay buffer of older data, which still allows us to drastically reduce the size of the overall training dataset, and hence model training time.

2024-05-31

Proceedings of the 46th International Conference on Software Engineering: Software Engineering in Practice (publié)

arxiv.org

Faithfulness Measurable Masked Language Models

Andreas Madsen

Siva Reddy

2024-05-01

ICML.cc/2024/Conference (spotlight)

Lookbehind-SAM: k steps back, 1 step forward

Goncalo Mordido

Pranshu Malviya

Aristide Baratin

2024-05-01

ICML.cc/2024/Conference (poster)

Towards Practical Tool Usage for Continually Learning LLMs

Jerry Huang

Prasanna Parthasarathi

Mehdi Rezagholizadeh

Large language models (LLMs) show an innate skill for solving language based tasks. But insights have suggested an inability to adjust for i… (voir plus)nformation or task-solving skills becoming outdated, as their knowledge, stored directly within their parameters, remains static in time. Tool use helps by offloading work to systems that the LLM can access through an interface, but LLMs that use them still must adapt to nonstationary environments for prolonged use, as new tools can emerge and existing tools can change. Nevertheless, tools require less specialized knowledge, therefore we hypothesize they are better suited for continual learning (CL) as they rely less on parametric memory for solving tasks and instead focus on learning when to apply pre-defined tools. To verify this, we develop a synthetic benchmark and follow this by aggregating existing NLP tasks to form a more realistic testing scenario. While we demonstrate scaling model size is not a solution, regardless of tool usage, continual learning techniques can enable tool LLMs to both adapt faster while forgetting less, highlighting their potential as continual learners.

2024-04-14

ArXiv (prépublication)

arxiv.org

Intelligent Switching for Reset-Free RL

Darshan Patil

Janarthanan Rajendran

Glen Berseth

In the real world, the strong episode resetting mechanisms that are needed to train agents in simulation are unavailable. The \textit{resett… (voir plus)ing} assumption limits the potential of reinforcement learning in the real world, as providing resets to an agent usually requires the creation of additional handcrafted mechanisms or human interventions. Recent work aims to train agents (\textit{forward}) with learned resets by constructing a second (\textit{backward}) agent that returns the forward agent to the initial state. We find that the termination and timing of the transitions between these two agents are crucial for algorithm success. With this in mind, we create a new algorithm, Reset Free RL with Intelligently Switching Controller (RISC) which intelligently switches between the two agents based on the agent's confidence in achieving its current goal. Our new method achieves state-of-the-art performance on several challenging environments for reset-free RL.

2024-01-16

ICLR.cc/2024/Conference (poster)

Intelligent Switching for Reset-Free RL

Darshan Patil

Janarthanan Rajendran

Glen Berseth

2024-01-16

ICLR.cc/2024/Conference (poster)

Mastering Memory Tasks with World Models

Mohammad Reza Samsami

Artem Zholus

Janarthanan Rajendran

Current model-based reinforcement learning (MBRL) agents struggle with long-term dependencies. This limits their ability to effectively solv… (voir plus)e tasks involving extended time gaps between actions and outcomes, or tasks demanding the recalling of distant observations to inform current actions. To improve temporal coherence, we integrate a new family of state space models (SSMs) in world models of MBRL agents to present a new method, Recall to Imagine (R2I). This integration aims to enhance both long-term memory and long-horizon credit assignment. Through a diverse set of illustrative tasks, we systematically demonstrate that R2I not only establishes a new state-of-the-art for challenging memory and credit assignment RL tasks, such as BSuite and POPGym, but also showcases superhuman performance in the complex memory domain of Memory Maze. At the same time, it upholds comparable performance in classic RL tasks, such as Atari and DMC, suggesting the generality of our method. We also show that R2I is faster than the state-of-the-art MBRL method, DreamerV3, resulting in faster wall-time convergence.

2024-01-16

ICLR.cc/2024/Conference (présentation orale)

Are self-explanations from Large Language Models faithful?

Andreas Madsen

Siva Reddy

2024-01-15

ArXiv (prépublication)

arxiv.org

Learning Conditional Policies for Crystal Design Using Offline Reinforcement Learning

Prashant Govindarajan

Santiago Miret

Jarrid Rector-Brooks

Mariano Phielipp

Janarthanan Rajendran

Navigating through the exponentially large chemical space to search for desirable materials is an extremely challenging task in material dis… (voir plus)covery. Recent developments in generative and geometric deep learning have shown...

2024-01-01

Digital Discovery (publié)

Fairness-Aware Structured Pruning in Transformers

A. Zayed

Goncalo Mordido

Samira Shabanian

Ioana Baldini