Glen Berseth

alexandre.brown@mila.quebec

Biographie

Glen Berseth est professeur agrégé au Département d'informatique et de recherche opérationnelle (DIRO) de l'Université de Montréal, membre académique principal de Mila – Institut québécois d'intelligence artificielle, détenteur d’une chaire en IA Canada-CIFAR et codirecteur du Laboratoire de robotique et d’IA intégrative de Montréal (REAL). Il a été chercheur postdoctoral à Berkeley Artificial Intelligence Research (BAIR), où il a travaillé avec Sergey Levine. Ses recherches portent sur la résolution de problèmes de prise de décision séquentielle (planification) pour les systèmes d'apprentissage autonomes du monde réel (robots). Elles ont couvert les domaines de la collaboration humain-robot, du renforcement, ainsi que de l'apprentissage continu, multiagent et hiérarchique et du méta-apprentissage. Glen Berseth a fait paraître des articles dans les meilleures publications des domaines de la robotique, de l'apprentissage automatique et de l'animation informatique. Il donne également un cours sur l'apprentissage des robots à l'Université de Montréal et à Mila, couvrant les recherches les plus récentes sur les techniques d'apprentissage automatique pour la création de robots généralistes.

Étudiants actuels

Özgür Aslan

Doctorat - UdeM

ozgur.aslan@mila.quebec

Doctorat - UdeM

Collaborateur·rice de recherche - Waterloo

ccloutie@uwaterloo.ca

Roger Creus-Castanyer

Doctorat - UdeM

Co-superviseur⋅e :

Pablo Samuel Castro

roger.creus-castanyer@mila.quebec

Doctorat - McGill

Superviseur⋅e principal⋅e :

Hsiu-Chin Lin

elham.daneshmand@mila.quebec

Léa Demeule

Doctorat - UdeM

Co-superviseur⋅e :

Chris Pal

lea.demeule@mila.quebec

kumaraditya.gupta@mila.quebec

Kumaraditya Gupta

Doctorat

Superviseur⋅e principal⋅e :

Liam Paull

Collaborateur·rice de recherche - UdeM

homayoun.honari@mila.quebec

Naser Kazemi

Collaborateur·rice de recherche - EPFL

naser.kazemi@mila.quebec

Adriana Knatchbull-Hugessen

Doctorat - UdeM

adriana.knatchbull-hugessen@mila.quebec

artur.kuramshin@mila.quebec

Artur Kuramshin

Maîtrise recherche - UdeM

daniel.lawson@mila.quebec

Daniel Lawson

Doctorat - UdeM

Co-superviseur⋅e :

Khimya Khetarpal

Postdoctorat - UdeM

tabitha-edith.lee@mila.quebec

Maîtrise recherche - UdeM

faisal.mohamed@mila.quebec

Stagiaire de recherche - UdeM

josuesmjr.mongan@gmail.com

Esra'a Saleh

Doctorat - UdeM

Co-superviseur⋅e :

Aaron Courville

esraa.saleh@mila.quebec

jesse@jessesilverberg.com

Jesse Silverberg

Doctorat - UdeM

Co-superviseur⋅e :

Marc Gendron-Bellemare

Doctorat - UdeM

Co-superviseur⋅e :

Yoshua Bengio

siddarth.venkatraman@mila.quebec

L'apprentissage par renforcement en temps réel

Albert Zhan

Doctorat - UdeM

albert.zhan@mila.quebec

Billets de blogue

Deux robots dans une cuisine, en train de préparer le dîner. L'un coupe les légumes et l'autre fait une omelette.

20 juin 2025

par

Ivan Anokhin

Matthew Riemer

Rishav Rishav

Gopeshh Subbaraj

Glen Berseth

Lire l'article

Fully Autonomous Real-World Reinforcement Learning with Applications to Mobile Manipulation

15 février 2023

Apprentissage par renforcement entièrement autonome dans le monde réel avec des applications à la manipulation mobile

par

Jędrzej Orbik

Charles Sun

Coline Devin

Glen Berseth

Lire l'article

Publications

Closing the Gap between TD Learning and Supervised Learning - A Generalisation Point of View

Raj Ghugare

Matthieu Geist

Benjamin Eysenbach

Some reinforcement learning (RL) algorithms can stitch pieces of experience to solve a task never seen before during training. This oft-soug… (voir plus)ht property is one of the few ways in which RL methods based on dynamic-programming differ from RL methods based on supervised-learning (SL). Yet, certain RL methods based on off-the-shelf SL algorithms achieve excellent results without an explicit mechanism for stitching; it remains unclear whether those methods forgo this important stitching property. This paper studies this question for the problems of achieving a target goal state and achieving a target return value. Our main result is to show that the stitching property corresponds to a form of combinatorial generalization: after training on a distribution of (state, goal) pairs, one would like to evaluate on (state, goal) pairs not seen together in the training data. Our analysis shows that this sort of generalization is different from i.i.d. generalization. This connection between stitching and generalisation reveals why we should not expect SL-based RL methods to perform stitching, even in the limit of large datasets and models. Based on this analysis, we construct new datasets to explicitly test for this property, revealing that SL-based methods lack this stitching property and hence fail to perform combinatorial generalization. Nonetheless, the connection between stitching and combinatorial generalisation also suggests a simple remedy for improving generalisation in SL: data augmentation. We propose a temporal data augmentation and demonstrate that adding it to SL-based methods enables them to successfully complete tasks not seen together during training. On a high level, this connection illustrates the importance of combinatorial generalization for data efficiency in time-series data beyond tasks beyond RL, like audio, video, or text.

2024-01-15

ICLR.cc/2024/Conference (poster)

Improving Intrinsic Exploration by Creating Stationary Objectives

Roger Creus Castanyer

Joshua Romoff

2024-01-15

ICLR.cc/2024/Conference (poster)

Searching for High-Value Molecules Using Reinforcement Learning and Transformers

Raj Ghugare

Santiago Miret

Adriana Hugessen

Mariano Phielipp

2024-01-15

ICLR.cc/2024/Conference (poster)

Enhancing Agent Learning through World Dynamics Modeling

Xingdi Yuan

Large language models (LLMs) have been increasingly applied to tasks in language understanding and interactive decision-making, with their i… (voir plus)mpressive performance largely attributed to the extensive domain knowledge embedded within them. However, the depth and breadth of this knowledge can vary across domains. Many existing approaches assume that LLMs possess a comprehensive understanding of their environment, often overlooking potential gaps in their grasp of actual world dynamics. To address this, we introduce Discover, Verify, and Evolve (DiVE), a framework that discovers world dynamics from a small number of demonstrations, verifies the accuracy of these dynamics, and evolves new, advanced dynamics tailored to the current situation. Through extensive evaluations, we assess the impact of each component on performance and compare the dynamics generated by DiVE to human-annotated dynamics. Our results show that LLMs guided by DiVE make more informed decisions, achieving rewards comparable to human players in the Crafter environment and surpassing methods that require prior task-specific training in the MiniHack environment.

2023-12-31

Findings of the Association for Computational Linguistics: EMNLP (publié)

Maximum State Entropy Exploration using Predecessor and Successor Representations

Animals have a developed ability to explore that aids them in important tasks such as locating food, exploring for shelter, and finding misp… (voir plus)laced items. These exploration skills necessarily track where they have been so that they can plan for finding items with relative efficiency. Contemporary exploration algorithms often learn a less efficient exploration strategy because they either condition only on the current state or simply rely on making random open-loop exploratory moves. In this work, we propose

2023-12-11

Conference on Neural Information Processing Systems (Accept (poster))

Adaptive Resolution Residual Networks

Léa Demeule

Mahtab Sandhu

We introduce Adaptive Resolution Residual Networks (ARRNs), a form of neural operator that enables the creation of networks for signal-based… (voir plus) tasks that can be rediscretized to suit any signal resolution. ARRNs are composed of a chain of Laplacian residuals that each contain ordinary layers, which do not need to be rediscretizable for the whole network to be rediscretizable. ARRNs have the property of requiring a lower number of Laplacian residuals for exact evaluation on lower-resolution signals, which greatly reduces computational cost. ARRNs also implement Laplacian dropout, which encourages networks to become robust to low-bandwidth signals. ARRNs can thus be trained once at high-resolution and then be rediscretized on the fly at a suitable resolution with great robustness.

2023-10-30

NeurIPS.cc/2023/Workshop/DLDE (spotlight)

Bootstrapping Adaptive Human-Machine Interfaces with Offline Reinforcement Learning

Jensen Gao

Siddharth Reddy

Anca Dragan

Sergey Levine

Adaptive interfaces can help users perform sequential decision-making tasks like robotic teleoperation given noisy, high-dimensional command… (voir plus) signals (e.g., from a brain-computer interface). Recent advances in human-in-the-loop machine learning enable such systems to improve by interacting with users, but tend to be limited by the amount of data that they can collect from individual users in practice. In this paper, we propose a reinforcement learning algorithm to address this by training an interface to map raw command signals to actions using a combination of offline pre-training and online fine-tuning. To address the challenges posed by noisy command signals and sparse rewards, we develop a novel method for representing and inferring the user's long-term intent for a given trajectory. We primarily evaluate our method's ability to assist users who can only communicate through noisy, high-dimensional input channels through a user study in which 12 participants performed a simulated navigation task by using their eye gaze to modulate a 128-dimensional command signal from their webcam. The results show that our method enables successful goal navigation more often than a baseline directional interface, by learning to denoise user commands signals and provide shared autonomy assistance. We further evaluate on a simulated Sawyer pushing task with eye gaze control, and the Lunar Lander game with simulated user commands, and find that our method improves over baseline interfaces in these domains as well. Extensive ablation experiments with simulated user commands empirically motivate each component of our method.

2023-09-30

2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (publié)

Torque-Based Deep Reinforcement Learning for Task-and-Robot Agnostic Learning on Bipedal Robots Using Sim-to-Real Transfer

Donghyeon Kim

Mathew Schwartz

Jaeheung Park

In this letter, we review the question of which action space is best suited for controlling a real biped robot in combination with Sim2Real … (voir plus)training. Position control has been popular as it has been shown to be more sample efficient and intuitive to combine with other planning algorithms. However, for position control, gain tuning is required to achieve the best possible policy performance. We show that, instead, using a torque-based action space enables task-and-robot agnostic learning with less parameter tuning and mitigates the sim-to-reality gap by taking advantage of torque control's inherent compliance. Also, we accelerate the torque-based-policy training process by pre-training the policy to remain upright by compensating for gravity. The letter showcases the first successful sim-to-real transfer of a torque-based deep reinforcement learning policy on a real human-sized biped robot.

2023-09-30

IEEE Robotics and Automation Letters (publié)

Towards Learning to Imitate from a Single Video Demonstration

Florian Golemo

Christopher Pal

Agents that can learn to imitate given video observation -- \emph{without direct access to state or action information} are more applicable … (voir plus)to learning in the natural world. However, formulating a reinforcement learning (RL) agent that facilitates this goal remains a significant challenge. We approach this challenge using contrastive training to learn a reward function comparing an agent's behaviour with a single demonstration. We use a Siamese recurrent neural network architecture to learn rewards in space and time between motion clips while training an RL policy to minimize this distance. Through experimentation, we also find that the inclusion of multi-task data and additional image encoding losses improve the temporal consistency of the learned rewards and, as a result, significantly improves policy learning. We demonstrate our approach on simulated humanoid, dog, and raptor agents in 2D and a quadruped and a humanoid in 3D. We show that our method outperforms current state-of-the-art techniques in these environments and can learn to imitate from a single video demonstration.

2023-07-24

International Conference on Machine Learning (Accept (Poster))

Robust and Versatile Bipedal Jumping Control through Reinforcement Learning

Zhongyu Li

Xue Bin Peng

Pieter Abbeel

Sergey Levine

Koushil Sreenath

2023-07-09

Robotics: Science and Systems XIX (publié)

Heterogeneous Crowd Simulation Using Parametric Reinforcement Learning.

Kaidong Hu

M. Brandon Haworth

Vladimir Pavlovic

Petros Faloutsos

Mubbasir Kapadia

Agent-based synthetic crowd simulation affords the cost-effective large-scale simulation and animation of interacting digital humans. Model-… (voir plus)based approaches have successfully generated a plethora of simulators with a variety of foundations. However, prior approaches have been based on statically defined models predicated on simplifying assumptions, limited video-based datasets, or homogeneous policies. Recent works have applied reinforcement learning to learn policies for navigation. However, these approaches may learn static homogeneous rules, are typically limited in their generalization to trained scenarios, and limited in their usability in synthetic crowd domains. In this article, we present a multi-agent reinforcement learning-based approach that learns a parametric predictive collision avoidance and steering policy. We show that training over a parameter space produces a flexible model across crowd configurations. That is, our goal-conditioned approach learns a parametric policy that affords heterogeneous synthetic crowds. We propose a model-free approach without centralization of internal agent information, control signals, or agent communication. The model is extensively evaluated. The results show policy generalization across unseen scenarios, agent parameters, and out-of-distribution parameterizations. The learned model has comparable computational performance to traditional methods. Qualitatively the model produces both expected (laminar flow, shuffling, bottleneck) and unexpected (side-stepping) emergent qualitative behaviours, and quantitatively the approach is performant across measures of movement quality.

2022-12-31

IEEE Transactions on Visualization and Computer Graphics (publié)

Robust and Versatile Bipedal Jumping Control through Multi-Task Reinforcement Learning

Zhongyu Li

Xue Bin Peng

Pieter Abbeel

Sergey Levine

Koushil Sreenath

2022-12-31

arXiv.org (prépublication)