Laurent Charlin

Biography

Laurent Charlin is a Canada CIFAR AI Chair at Mila and an associate professor at HEC, the business school affiliated with the University de Montréal. He is also a core member of Mila—Quebec Institute for Artificial Intelligence.

Charlin’s research focuses on developing novel machine learning models to aid in decision-making. Recent work has focused on learning from data that changes over time, and on applications in fields such as recommender systems and optimization.

He has a number of highly cited publications on dialogue systems (chatbots). He co-developed the Toronto Paper Matching System (TPMS), which has been widely used by computer science conferences for matching reviewers to papers. He has also given MOOCs, introductory talks and media interviews to contribute to knowledge transfer and improve AI literacy.

Current Students

Neda Adl

Master's Research - HEC Montréal

Neda@mila.quebec

Anirudh Buvanesh

PhD - Université de Montréal

Co-supervisor :

Aaron Courville

anirudb1102@gmail.com

felix.gauthier@mila.quebec

Félix Gauthier

Master's Research - HEC Montréal

Soraya Ghassemlou

Master's Research - McGill University

soraya.ghassemlou@mila.quebec

Website

nicolas.goulet@mila.quebec

Nicolas Goulet

PhD - HEC Montréal

Principal supervisor :

Tegan Maharaj

shubham.gupta@mila.quebec

Shubham Gupta

PhD - Université Laval

Principal supervisor :

Cem Subakan

Ben Hudson

PhD - Université de Montréal

Co-supervisor :

Emma Frejinger

ben.hudson@mila.quebec

Website

marc.morissette@gmail.com

Marc-Andre Morissette

Master's Research - Université de Montréal

mizu.nishikawa-toomey@mila.quebec

Mizu Nishikawa-Toomey

PhD - Université de Montréal

Co-supervisor :

Dhanya Sridhar

PhD - Concordia University

Principal supervisor :

Mirco Ravanelli

firat.oncel@mila.quebec

Collaborating Alumni - Université de Montréal

ostapeno@mila.quebec

emiliano.penaloza@mila.quebec

Emiliano Penaloza

PhD - Université de Montréal

Website

omar.salemohamed@mila.quebec

Gaurav Sahu

Postdoctorate - HEC Montréal

Co-supervisor :

Chris Pal

gaurav.sahu@mila.quebec

PhD - Université de Montréal

Yipeng Zhang

PhD - Université de Montréal

yipeng.zhang@mila.quebec

Publications

Bayesian learning of Causal Structure and Mechanisms with GFlowNets and Variational Bayes

Mizu Nishikawa-Toomey

Tristan Deleu

Jithendaraa Subramanian

Yoshua Bengio

Bayesian causal structure learning aims to learn a posterior distribution over directed acyclic graphs (DAGs), and the mechanisms that defin… (see more)e the relationship between parent and child variables. By taking a Bayesian approach, it is possible to reason about the uncertainty of the causal model. The notion of modelling the uncertainty over models is particularly crucial for causal structure learning since the model could be unidentifiable when given only a finite amount of observational data. In this paper, we introduce a novel method to jointly learn the structure and mechanisms of the causal model using Variational Bayes, which we call Variational Bayes-DAG-GFlowNet (VBG). We extend the method of Bayesian causal structure learning using GFlowNets to learn not only the posterior distribution over the structure, but also the parameters of a linear-Gaussian model. Our results on simulated data suggest that VBG is competitive against several baselines in modelling the posterior over DAGs and mechanisms, while offering several advantages over existing methods, including the guarantee to sample acyclic graphs, and the flexibility to generalize to non-linear causal mechanisms.

2022-11-03

arXiv (preprint)

arxiv.org

Attention for Compositional Modularity

Oleksiy Ostapenko

Pau Rodríguez

Alexandre Lacoste

Modularity and compositionality are promising inductive biases for addressing longstanding problems in machine learning such as better syste… (see more)matic generalization, as well as better transfer and lower forgetting in the context of continual learning. Here we study how attention-based module selection can help achieve composi-tonal modularity – i.e. decomposition of tasks into meaningful sub-tasks which are tackled by independent architectural entities that we call modules. These sub-tasks must be reusable and the system should be able to learn them without additional supervision. We design a simple experimental setup in which the model is trained to solve mathematical equations with multiple math operations applied sequentially. We study different attention-based module selection strategies, inspired by the principles introduced in the recent literature. We evaluate the method’s ability to learn modules that can recover the underling sub-tasks (operation) used for data generation, as well as the ability to generalize compositionally. We find that meaningful module selection (i.e. routing) is the key to compositional generalization. Further, without access to the privileged information about which part of the input should be used for module selection, the routing component performs poorly for samples that are compositionally out of training distribution. We find that the the main reason for this lies in the routing component, since many of the tested methods perform well OOD if we report the performance of the best performing path at test time. Additionally, we study the role of the number of primitives, the number of training points and bottlenecks for modular specialization.

2022-10-19

NeurIPS.cc/2022/Workshop/Attention (poster)

The Machine Learning for Combinatorial Optimization Competition (ML4CO): Results and Insights

Maxime Gasse

Quentin Cappart

Jonas Charfreitag

Didier Chételat

Antonia Chmiela

Justin Dumouchelle

Ambros Gleixner

Aleksandr M. Kazachkov

Elias Khalil

Pawel Lichocki

Andrea Lodi

Miles Lubin

Chris J. Maddison

Christopher Morris

Dimitri J. Papageorgiou

Augustin Parjadis

Sebastian Pokutta

Antoine Prouvost

Lara Scavuzzo … (see 21 more)

Giulia Zarpellon

Linxin Yang

Sha Lai

Akang Wang

Xiaodong Luo

Xiang Zhou

Haohan Huang

Shengcheng Shao

Yuanming Zhu

Dong Zhang

Tao Quan

Zixuan Cao

Yang Xu

Zhewei Huang

Shuchang Zhou

Chen Binbin

He Minggui

Hao Hao

Zhang Zhiyu

An Zhiwu

Mao Kun

Combinatorial optimization is a well-established area in operations research and computer science. Until recently, its methods have focused … (see more)on solving problem instances in isolation, ignoring that they often stem from related data distributions in practice. However, recent years have seen a surge of interest in using machine learning as a new approach for solving combinatorial problems, either directly as solvers or by enhancing exact solvers. Based on this context, the ML4CO aims at improving state-of-the-art combinatorial optimization solvers by replacing key heuristic components. The competition featured three challenging tasks: finding the best feasible solution, producing the tightest optimality certificate, and giving an appropriate solver configuration. Three realistic datasets were considered: balanced item placement, workload apportionment, and maritime inventory routing. This last dataset was kept anonymous for the contestants.

2022-07-19

Proceedings of the NeurIPS 2021 Competitions and Demonstrations Track (published)

proceedings.mlr.press

Learning to Cut by Looking Ahead: Cutting Plane Selection via Imitation Learning

Max B. Paulus

Giulia Zarpellon

Andreas Krause

Chris J. Maddison

Cutting planes are essential for solving mixed-integer linear problems (MILPs), because they facilitate bound improvements on the optimal so… (see more)lution value. For selecting cuts, modern solvers rely on manually designed heuristics that are tuned to gauge the potential effectiveness of cuts. We show that a greedy selection rule explicitly looking ahead to select cuts that yield the best bound improvement delivers strong decisions for cut selection - but is too expensive to be deployed in practice. In response, we propose a new neural architecture (NeuralCut) for imitation learning on the lookahead expert. Our model outperforms standard baselines for cut selection on several synthetic MILP benchmarks. Experiments with a B&C solver for neural network verification further validate our approach, and exhibit the potential of learning methods in this setting.

2022-07-18

International Conference on Machine Learning (Accept for Short Presentation)

proceedings.mlr.press

IG-RL: Inductive Graph Reinforcement Learning for Massive-Scale Traffic Signal Control

FranÃ§ois-Xavier Devailly

Denis Larocque

Scaling adaptive traffic signal control involves dealing with combinatorial state and action spaces. Multi-agent reinforcement learning atte… (see more)mpts to address this challenge by distributing control to specialized agents. However, specialization hinders generalization and transferability, and the computational graphs underlying neural-network architectures—dominating in the multi-agent setting—do not offer the flexibility to handle an arbitrary number of entities which changes both between road networks, and over time as vehicles traverse the network. We introduce Inductive Graph Reinforcement Learning (IG-RL) based on graph-convolutional networks which adapts to the structure of any road network, to learn detailed representations of traffic signal controllers and their surroundings. Our decentralized approach enables learning of a transferable-adaptive-traffic-signal-control policy. After being trained on an arbitrary set of road networks, our model can generalize to new road networks and traffic distributions, with no additional training and a constant number of parameters, enabling greater scalability compared to prior methods. Furthermore, our approach can exploit the granularity of available data by capturing the (dynamic) demand at both the lane level and the vehicle level. The proposed method is tested on both road networks and traffic settings never experienced during training. We compare IG-RL to multi-agent reinforcement learning and domain-specific baselines. In both synthetic road networks and in a larger experiment involving the control of the 3,971 traffic signals of Manhattan, we show that different instantiations of IG-RL outperform baselines.

2022-06-30

IEEE Transactions on Intelligent Transportation Systems (published)

arxiv.org

A New Era: Intelligent Tutoring Systems Will Transform Online Learning for Millions

Francois St-Hilaire

Dung D. Vu

Antoine Frau

Nathan J. Burns

Farid Faraji

Joseph Potochny

Stephane Robert

Arnaud Roussel

Selene Zheng

Taylor Glazier

Junfel Vincent Romano

Robert Belfer

Muhammad Shayan

Ariella Smofsky

Tommy Delarosbil

Seulmin Ahn

Simon Eden-Walker

Kritika Sony

Ansona Onyi Ching

Sabina Elkins … (see 11 more)

A. Stepanyan

Adela Matajova

Victor Chen

Hossein Sahraei

Robert Larson

N. Markova

Andrew Barkett

Yoshua Bengio

Iulian V. Serban

Ekaterina Kochmar

2022-03-02

ArXiv (preprint)

arxiv.org

COIL: A Deep Architecture for Column Generation

Behrouz Babaki

Sanjay Dominik Jena

. Column generation is a popular method to solve large-scale linear programs with an exponential number of variables. Several important appl… (see more)ications, such as the vehicle routing problem, rely on this technique in order to be solved. However, in practice, column generation methods suffer from slow convergence (i.e. they require too many iterations). Stabilization techniques, which carefully select the column to add at each iteration, are commonly used to improve convergence. In this work, we frame the problem of selecting which columns to add as one of sequential decision-making. We propose a neural column generation architecture that iteratively selects columns to be added to the problem. Our architecture is inspired by stabilization techniques and predicts the optimal duals, which are then used to select the columns to add. We proposed architecture, trained using imitation learning. Exemplified on the Vehicle Routing Problem, we show that several machine learning models yield good performance in predicting the optimal duals and that our architecture outperforms them as well as a popular state-of-the-art stabilization technique. Further, the architecture approach can generalize to instances larger than those observed during training.

2021-12-31

(published)

www.semanticscholar.org

Task-Agnostic Continual Reinforcement Learning: In Praise of a Simple Baseline

Massimo Caccia

Jonas Mueller

Taesup Kim

Rasool Fakoor

We study task-agnostic continual reinforcement learning (TACRL) in which standard RL challenges are compounded with partial observability st… (see more)emming from task agnosticism, as well as additional difﬁculties of continual learning (CL), i.e., learning on a non-stationary sequence of tasks. Here we compare TACRL methods with their soft upper bounds prescribed by previous literature: multi-task learning (MTL) methods which do not have to deal with non-stationary data distributions, as well as task-aware methods, which are allowed to operate under full observability . We consider a previously unexplored and straightforward baseline for TACRL, replay-based recurrent RL (3RL), in which we augment an RL algorithm with recurrent mechanisms to address partial observability and experience replay mechanisms to address catastrophic forgetting in CL. Studying empirical performance in a sequence of RL tasks, we ﬁnd surprising occurrences of 3RL matching and overcoming the MTL and task-aware soft upper bounds. We lay out hypotheses that could explain this inﬂection point of continual and task-agnostic learning research. Our hypotheses are empirically tested in continuous control tasks via a large-scale study of the popular multi-task and continual learning benchmark Meta-World. By analyzing different training statistics including gradient conﬂict, we ﬁnd evidence that 3RL’s outperformance stems from its ability to quickly infer how new tasks relate with the previous ones, enabling forward transfer.

2021-12-31

arXiv.org (preprint)

Towards Policy-Guided Conversational Recommendation with Dialogue Acts

Paul Crook

Y-Lan Boureau

J. Weston

Akbar Karimi

Leonardo Rossi

Andrea Prati

Wenqiang Lei

Xiangnan He

Qingyun Yisong Miao

Richang Wu

Min-Yen Hong

Kan Tat-Seng

Raymond Li

Samira Ebrahimi Kahou

Hannes Schulz

Zujie Liang

Huang Hu

Can Xu

Jian Miao

Lizi Liao … (see 47 more)

Ryuichi Takanobu

Yunshan Ma

Xun Yang

Wenchang Ma

Minlie Huang

Minghao Tu

Iulian Serban

Aaron C. Courville

David Silver

Julian Schrittwieser

K. Simonyan

Ioannis Antonoglou

Aja Huang

A. Guez

Hanlin Zhu

O. Vinyals

Igor Babuschkin

Junyoung Chung

M. Mathieu

Max Jaderberg

Wojciech M. Czar-725 necki

A. Dudzik

Petko Georgiev

Richard Powell

T. Ewalds

Dan Horgan

M. Kroiss

Ivo Danihelka

J. Agapiou

Junhyuk Oh

Valentin Dalibard

David Choi

L. Sifre

Yury Sulsky

Sasha Vezhnevets

James Molloy

Trevor Cai

D. Budden

T. Paine

Caglar Gulçehre

Ziyu Wang

Tobias Pfaff

Tobias Pohlen

2021-12-31

(published)

www.semanticscholar.org

Neural Column Generation for Capacitated Vehicle Routing

Behrouz Babaki

Sanjay Dominik Jena

The column generation technique is essential for solving linear programs with an exponential number of variables. Many important application… (see more)s such as the vehicle routing problem (VRP) now require it. However, in practice, getting column generation to converge is challenging. It often ends up adding too many columns. In this work, we frame the problem of selecting which columns to add as one of sequential decision-making. We propose a neural column generation architecture that iteratively selects columns to be added to the problem. The architecture, inspired by stabilization techniques, first predicts the optimal duals. These predictions are then used to obtain the columns to add. We show using VRP instances that in this setting several machine learning models yield good performance on the task and that our proposed architecture learned using imitation learning outperforms a modern stabilization technique.

2021-12-15

AAAI.org/2022/Workshop/ML4OR-22 (poster)

Continual Learning via Local Module Composition

Oleksiy Ostapenko

Pau Rodríguez

Massimo Caccia

Modularity is a compelling solution to continual learning (CL), the problem of modeling sequences of related tasks. Learning and then compos… (see more)ing modules to solve different tasks provides an abstraction to address the principal challenges of CL including catastrophic forgetting, backward and forward transfer across tasks, and sub-linear model growth. We introduce local module composition (LMC), an approach to modular CL where each module is provided a local structural component that estimates a module's relevance to the input. Dynamic module composition is performed layer-wise based on local relevance scores. We demonstrate that agnosticity to task identities (IDs) arises from (local) structural learning that is module-specific as opposed to the task- and/or model-specific as in previous works, making LMC applicable to more CL settings compared to previous works. In addition, LMC also tracks statistics about the input distribution and adds new modules when outlier samples are detected. In the first set of experiments, LMC performs favorably compared to existing methods on the recent Continual Transfer-learning Benchmark without requiring task identities. In another study, we show that the locality of structural learning allows LMC to interpolate to related but unseen tasks (OOD), as well as to compose modular networks trained independently on different task sequences into a third modular network without any fine-tuning. Finally, in search for limitations of LMC we study it on more challenging sequences of 30 and 100 tasks, demonstrating that local module selection becomes much more challenging in presence of a large number of candidate modules. In this setting best performing LMC spawns much fewer modules compared to an oracle based baseline, however, it reaches a lower overall accuracy. The codebase is available under https://github.com/oleksost/LMC.

2021-12-06

Conference on Neural Information Processing Systems (Accept (Poster))

Pretraining Representations for Data-Efficient Reinforcement Learning

Philip Bachman

Data efficiency is a key challenge for deep reinforcement learning. We address this problem by using unlabeled data to pretrain an encoder w… (see more)hich is then finetuned on a small amount of task-specific data. To encourage learning representations which capture diverse aspects of the underlying MDP, we employ a combination of latent dynamics modelling and unsupervised goal-conditioned RL. When limited to 100k steps of interaction on Atari games (equivalent to two hours of human experience), our approach significantly surpasses prior work combining offline representation pretraining with task-specific finetuning, and compares favourably with other pretraining methods that require orders of magnitude more data. Our approach shows particular promise when combined with larger models as well as more diverse, task-aligned observational data -- approaching human-level performance and data-efficiency on Atari in our best setting. We provide code associated with this work at https://github.com/mila-iqia/SGI.

2021-12-06

Conference on Neural Information Processing Systems (Accept (Poster))