Portrait of Pierre-Luc Bacon

Pierre-Luc Bacon

Core Academic Member
pierre-luc.bacon@mila.quebec
Canada CIFAR AI Chair
Assistant Professor, Université de Montréal, Department of Computer Science and Operations Research
Research Topics
Reinforcement Learning
Website
Google Scholar

Biography

Pierre-Luc Bacon is an assistant professor at Université de Montréal in the Department of Computer Science and Operations Research (DIRO). He is also a core academic member of Mila – Quebec Artificial Intelligence Institute and IVADO, and holds a Facebook CIFAR AI Chair. Bacon leads a research group that investigates the challenges posed by the curse of the horizon in reinforcement learning and optimal control.

Current Students

Diego Calanzone
Research Intern - Université de Montréal
Website
Github
Google Scholar
Léa Côté-Turcotte
Master's Research - Université de Montréal
Franciszek Czyzowicz
Research Intern - Université de Montréal
Pierluca D'Oro
PhD - Université de Montréal
Co-supervisor :
Marc Gendron-Bellemare
Website
Github
Google Scholar
Esther Derman
Postdoctorate - Université de Montréal
Co-supervisor :
Erick Delage
Website
Github
Google Scholar
Simon Dufort-Labbé
PhD - Université de Montréal
Mahan Fathi
Master's Research - Université de Montréal
Website
Github
Google Scholar
Arielle Gazzé
Master's Research - Université de Montréal
Clement Gehring
Postdoctorate - Université de Montréal
Niki Howe
PhD - Université de Montréal
Website
Github
Google Scholar
David Kanaa
Collaborating Alumni
Google Scholar
Lu Li
Collaborating researcher - Université de Montréal
Michel Ma
PhD - Université de Montréal
Github
Google Scholar
Sobhan Mohammadpour
Master's Research - Université de Montréal
Aneri Muni
PhD - Université de Montréal
Tianwei Ni
PhD - Université de Montréal
Website
Github
Google Scholar
Evgenii Nikishin
PhD - Université de Montréal
Co-supervisor :
Aaron Courville
Website
Github
Google Scholar
Anushree Rankawat
PhD - Université de Montréal
Samy Rasmy
Research Intern - Université de Montréal
Github
Julien Roy
PhD - Polytechnique Montréal
Principal supervisor :
Chris Pal
Github
Google Scholar
Vincent Taboga
Postdoctorate - Polytechnique Montréal
Principal supervisor :
Yoshua Bengio
Github
Google Scholar
Justin Veilleux
Research Intern - Université de Montréal

Publications

Double Gumbel Q-Learning.
David Yu-Tung Hui
Aaron Courville
Pierre-Luc Bacon
openreview.net
Policy Optimization in a Noisy Neighborhood: On Return Landscapes in Continuous Control
Nathan Rahn
Pierluca D'Oro
Harley Wiltzer
Pierre-Luc Bacon
Marc Gendron-Bellemare
Deep reinforcement learning agents for continuous control are known to exhibit significant instability in their performance over time. In th… (see more)is work, we provide a fresh perspective on these behaviors by studying the return landscape: the mapping between a policy and a return. We find that popular algorithms traverse noisy neighborhoods of this landscape, in which a single update to the policy parameters leads to a wide range of returns. By taking a distributional view of these returns, we map the landscape, characterizing failure-prone regions of policy space and revealing a hidden dimension of policy quality. We show that the landscape exhibits surprising structure by finding simple paths in parameter space which improve the stability of a policy. To conclude, we develop a distribution-aware procedure which finds such paths, navigating away from noisy neighborhoods in order to improve the robustness of a policy. Taken together, our results provide new insight into the optimization, evaluation, and design of agents.
openreview.net
When Do Transformers Shine in RL? Decoupling Memory from Credit Assignment
Tianwei Ni
Michel Ma
Benjamin Eysenbach
Pierre-Luc Bacon
Reinforcement learning (RL) algorithms face two distinct challenges: learning effective representations of past and present observations, an… (see more)d determining how actions influence future returns. Both challenges involve modeling long-term dependencies. The Transformer architecture has been very successful to solve problems that involve long-term dependencies, including in the RL domain. However, the underlying reason for the strong performance of Transformer-based RL methods remains unclear: is it because they learn effective memory, or because they perform effective credit assignment? After introducing formal definitions of memory length and credit assignment length, we design simple configurable tasks to measure these distinct quantities. Our empirical results reveal that Transformers can enhance the memory capability of RL algorithms, scaling up to tasks that require memorizing observations
openreview.net
Goal-conditioned GFlowNets for Controllable Multi-Objective Molecular Design
Julien Roy
Pierre-Luc Bacon
Chris Pal
Emmanuel Bengio
In recent years, in-silico molecular design has received much attention from the machine learning community. When designing a new compound f… (see more)or pharmaceutical applications, there are usually multiple properties of such molecules that need to be optimised: binding energy to the target, synthesizability, toxicity, EC50, and so on. While previous approaches have employed a scalarization scheme to turn the multi-objective problem into a preference-conditioned single objective, it has been established that this kind of reduction may produce solutions that tend to slide towards the extreme points of the objective space when presented with a problem that exhibits a concave Pareto front. In this work we experiment with an alternative formulation of goal-conditioned molecular generation to obtain a more controllable conditional model that can uniformly explore solutions along the entire Pareto front.
2023-06-23
ICML.cc/2023/Workshop/DeployableGenerativeAI (published)
doi.org
openreview.net
Sample-Efficient Reinforcement Learning by Breaking the Replay Ratio Barrier
Pierluca D'Oro
Max Schwarzer
Evgenii Nikishin
Pierre-Luc Bacon
Marc Gendron-Bellemare
Aaron Courville
Increasing the replay ratio, the number of updates of an agent's parameters per environment interaction, is an appealing strategy for improv… (see more)ing the sample efficiency of deep reinforcement learning algorithms. In this work, we show that fully or partially resetting the parameters of deep reinforcement learning agents causes better replay ratio scaling capabilities to emerge. We push the limits of the sample efficiency of carefully-modified algorithms by training them using an order of magnitude more updates than usual, significantly improving their performance in the Atari 100k and DeepMind Control Suite benchmarks. We then provide an analysis of the design choices required for favorable replay ratio scaling to be possible and discuss inherent limits and tradeoffs.
2023-02-01
ICLR.cc/2023/Conference (notable)
openreview.net
openreview.net
Block-State Transformers
Mahan Fathi
Jonathan Pilault
Pierre-Luc Bacon
Chris Pal
Orhan Firat
Ross Goroshin
2023-01-01
NeurIPS (published)
arxiv.org
arxiv.org
Options of Interest: Temporal Abstraction with Interest Functions
Khimya Khetarpal
Martin Klissarov
Maxime Chevalier-Boisvert
Pierre-Luc Bacon
Doina Precup
Temporal abstraction refers to the ability of an agent to use behaviours of controllers which act for a limited, variable amount of time. Th… (see more)e options framework describes such behaviours as consisting of a subset of states in which they can initiate, an internal policy and a stochastic termination condition. However, much of the subsequent work on option discovery has ignored the initiation set, because of difficulty in learning it from data. We provide a generalization of initiation sets suitable for general function approximation, by defining an interest function associated with an option. We derive a gradient-based learning algorithm for interest functions, leading to a new interest-option-critic architecture. We investigate how interest functions can be leveraged to learn interpretable and reusable temporal abstractions. We demonstrate the efficacy of the proposed approach through quantitative and qualitative results, in both discrete and continuous environments.
2020-04-03
Proceedings of the AAAI Conference on Artificial Intelligence (published)
doi.org
arxiv.org