Portrait of David Meger

David Meger

Associate Academic Member
Associate Professor, McGill University, School of Computer Science
Research Topics
Computer Vision
Reinforcement Learning

Biography

David Meger is an associate professor at McGill University’s School of Computer Science.

He co-directs the Mobile Robotics Lab within the Centre for Intelligent Machines, one of Canada's largest and longest-running robotics research groups. He was the general chair of Canada’s first joint CS-CAN conference in 2023.

Meger's research contributions include visually guided robots powered by active vision and learning, deep reinforcement learning models that are widely cited and used by researchers and industry worldwide, and field robotics that allow for autonomous deployment underwater and on land.

Current Students

William Bonilla
PhD - McGill University
Github
Valliappan Chidambaram Adaikkappan
PhD - McGill University
Github
Google Scholar
Wesley Chung
PhD - McGill University
Co-supervisor :
Doina Precup
Farnoosh Faraji
PhD - McGill University
Co-supervisor :
Gregory Dudek
Github
Google Scholar
Anas Houssaini
Master's Research - McGill University
Co-supervisor :
Hsiu-Chin Lin
Github
Zina Kamel
Master's Research - McGill University
Co-supervisor :
Hsiu-Chin Lin
Sahand Rezaei-Shoshtari
PhD - McGill University
Principal supervisor :
Doina Precup
Website
Github
Google Scholar
Arian Sargazi
PhD - McGill University
Junming(Clark) Shi
Master's Research - McGill University
Github
Steven Wang
Master's Research - McGill University
Harley Wiltzer
PhD - McGill University
Co-supervisor :
Marc Gendron-Bellemare
Website
Github
Google Scholar
Hanna Yurchyk
PhD - McGill University

Publications

Where Off-Policy Deep Reinforcement Learning Fails
Scott Fujimoto
David Meger
Doina Precup
This work examines batch reinforcement learning–the task of maximally exploiting a given batch of off-policy data, without further data co… (see more)llection. We demonstrate that due to errors introduced by extrapolation, standard off-policy deep reinforcement learning algorithms, such as DQN and DDPG, are only capable of learning with data correlated to their current policy, making them ineffective for most off-policy applications. We introduce a novel class of off-policy algorithms, batch-constrained reinforcement learning, which restricts the action space to force the agent towards behaving on-policy with respect to a subset of the given data. We extend this notion to deep reinforcement learning, and to the best of our knowledge, present the first continuous control deep reinforcement learning algorithm which can learn effectively from uncorrelated off-policy data.
2018-09-26
(published)
openreview.net
openreview.net
Addressing Function Approximation Error in Actor-Critic Methods
Scott Fujimoto
Herke van Hoof
David Meger
In value-based reinforcement learning methods such as deep Q-learning, function approximation errors are known to lead to overestimated valu… (see more)e estimates and suboptimal policies. We show that this problem persists in an actor-critic setting and propose novel mechanisms to minimize its effects on both the actor and the critic. Our algorithm builds on Double Q-learning, by taking the minimum value between a pair of critics to limit overestimation. We draw the connection between target networks and overestimation bias, and suggest delaying policy updates to reduce per-update error and further improve performance. We evaluate our method on the suite of OpenAI gym tasks, outperforming the state of the art in every environment tested.
2018-07-02
Proceedings of the 35th International Conference on Machine Learning (published)
proceedings.mlr.press
proceedings.mlr.press
Deep Reinforcement Learning that Matters
Peter Henderson
Riashat Islam
Philip Bachman
Joelle Pineau
Doina Precup
David Meger
In recent years, significant progress has been made in solving challenging problems across various domains using deep reinforcement learning… (see more) (RL). Reproducing existing work and accurately judging the improvements offered by novel methods is vital to sustaining this progress. Unfortunately, reproducing results for state-of-the-art deep RL methods is seldom straightforward. In particular, non-determinism in standard benchmark environments, combined with variance intrinsic to the methods, can make reported results tough to interpret. Without significance metrics and tighter standardization of experimental reporting, it is difficult to determine whether improvements over the prior state-of-the-art are meaningful. In this paper, we investigate challenges posed by reproducibility, proper experimental techniques, and reporting procedures. We illustrate the variability in reported metrics and results when comparing against common baselines and suggest guidelines to make future results in deep RL more reproducible. We aim to spur discussion about how to ensure continued progress in the field by minimizing wasted effort stemming from results that are non-reproducible and easily misinterpreted.
2018-04-28
Proceedings of the AAAI Conference on Artificial Intelligence (published)
doi.org
arxiv.org
OptionGAN: Learning Joint Reward-Policy Options using Generative Adversarial Inverse Reinforcement Learning
Peter Henderson
Wei-Di Chang
Pierre-Luc Bacon
David Meger
Joelle Pineau
Doina Precup
Reinforcement learning has shown promise in learning policies that can solve complex problems. However, manually specifying a good reward fu… (see more)nction can be difficult, especially for intricate tasks. Inverse reinforcement learning offers a useful paradigm to learn the underlying reward function directly from expert demonstrations. Yet in reality, the corpus of demonstrations may contain trajectories arising from a diverse set of underlying reward functions rather than a single one. Thus, in inverse reinforcement learning, it is useful to consider such a decomposition. The options framework in reinforcement learning is specifically designed to decompose policies in a similar light. We therefore extend the options framework and propose a method to simultaneously recover reward options in addition to policy options. We leverage adversarial methods to learn joint reward-policy options using only observed expert states. We show that this approach works well in both simple and complex continuous control tasks and shows significant performance increases in one-shot transfer learning.
2018-04-28
Proceedings of the AAAI Conference on Artificial Intelligence (published)
doi.org
arxiv.org