Portrait of Marc Gendron-Bellemare is unavailable

Marc Gendron-Bellemare

Core Industry Member
Canada CIFAR AI Chair
Associate Professor, McGill University, School of Computer Science
Adjunct Professor, Université de Montréal, Department of Computer Science and Operations Research
Chief Scientific Officer, Reliant AI
Research Topics
Reinforcement Learning
Representation Learning

Biography

I am Chief Scientific Officer at Reliant AI, an adjunct professor at the School of Computer and Science at McGill University, and an adjunct professor at the Department of Computer Science and Operations Research (DIRO) at Université de Montréal.

Previously, I was a research scientist at Google Brain in Montréal, where my research focused on reinforcement learning effort. From 2013 to 2017, I worked at DeepMind in the U.K. I received my PhD from the University of Alberta under the supervision of Michael Bowling and Joel Veness.

My research lies at the intersection of reinforcement learning and probabilistic prediction. I am also interested in deep learning, generative modelling, online learning and information theory.

Current Students

Adrien Ali Taiga
PhD - Université de Montréal
Principal supervisor :
Aaron Courville
Pierluca D'Oro
PhD - Université de Montréal
Principal supervisor :
Pierre-Luc Bacon
Website
Github
Google Scholar
Jesse Farebrother
PhD - McGill University
Co-supervisor :
Doina Precup
Website
Github
Google Scholar
Nate Rahn
PhD - McGill University
Co-supervisor :
Doina Precup
Max Schwarzer
PhD - Université de Montréal
Principal supervisor :
Aaron Courville
Harley Wiltzer
PhD - McGill University
Principal supervisor :
David Meger
Website
Github
Google Scholar

Publications

A Geometric Perspective on Optimal Representations for Reinforcement Learning
Marc Gendron-Bellemare
Will Dabney
Robert Dadashi
Adrien Ali Taiga
Pablo Samuel Castro
Nicolas Le Roux
Dale Schuurmans
Tor Lattimore
Clare Lyle
openreview.net
A Geometric Perspective on Optimal Representations for Reinforcement Learning
Marc Gendron-Bellemare
Will Dabney
Robert Dadashi
Adrien Ali Taiga
Pablo Samuel Castro
Nicolas Le Roux
Dale Schuurmans
Tor Lattimore
Clare Lyle
We propose a new perspective on representation learning in reinforcement learning based on geometric properties of the space of value functi… (see more)ons. We leverage this perspective to provide formal evidence regarding the usefulness of value functions as auxiliary tasks. Our formulation considers adapting the representation to minimize the (linear) approximation of the value function of all stationary policies for a given environment. We show that this optimization reduces to making accurate predictions regarding a special class of value functions which we call adversarial value functions (AVFs). We demonstrate that using value functions as auxiliary tasks corresponds to an expected-error relaxation of our formulation, with AVFs a natural candidate, and identify a close relationship with proto-value functions (Mahadevan, 2005). We highlight characteristics of AVFs and their usefulness as auxiliary tasks in a series of experiments on the four-room domain.
arxiv.org
Dopamine: A Research Framework for Deep Reinforcement Learning
Pablo Samuel Castro
Subhodeep Moitra
Carles Gelada
Saurabh Kumar
Marc Gendron-Bellemare
Deep reinforcement learning (deep RL) research has grown significantly in recent years. A number of software offerings now exist that provid… (see more)e stable, comprehensive implementations for benchmarking. At the same time, recent deep RL research has become more diverse in its goals. In this paper we introduce Dopamine, a new research framework for deep RL that aims to support some of that diversity. Dopamine is open-source, TensorFlow-based, and provides compact and reliable implementations of some state-of-the-art deep RL agents. We complement this offering with a taxonomy of the different research objectives in deep RL research. While by no means exhaustive, our analysis highlights the heterogeneity of research in the field, and the value of frameworks such as ours.
2018-09-27
ArXiv (preprint)
arxiv.org
arxiv.org