Liam Paull

Biographie

Liam Paull est professeur adjoint à l'Université de Montréal et codirige le Laboratoire de robotique et d’IA intégrative de Montréal (REAL). Son laboratoire se concentre sur les problèmes de robotique, y compris la construction de représentations du monde (pour la localisation et la cartographie simultanées, par exemple), la modélisation de l'incertitude et la construction de meilleurs flux de travail pour enseigner de nouvelles tâches aux agents robotiques (notamment par la simulation ou la démonstration). Auparavant, Liam Paull a été chercheur au Computer Science and Artificial Intelligence Laboratory (CSAIL) du Massachusetts Institute of Technology (MIT), où il a dirigé le projet de voiture autonome financé par le Toyota Research Institute (TRI). Il a également été chercheur postdoctoral au laboratoire de robotique marine du MIT, où il a travaillé sur la technique SLAM (Simultaneous Localization and Mapping) pour les robots sous-marins. Il a obtenu son doctorat en 2013 à l'Université du Nouveau-Brunswick : il s’y est intéressé à la planification robuste et adaptative pour les véhicules sous-marins. Il est cofondateur et directeur de la Fondation Duckietown, dont l'objectif est de rendre accessibles à tous·tes les expériences d'apprentissage de la robotique.

Étudiants actuels

Ria Arora

Maîtrise recherche - UdeM

Superviseur⋅e principal⋅e :

Guy Wolf

Adam Burhan

Maîtrise recherche - UdeM

Andrea Daniele

Collaborateur·rice de recherche - UdeM

Rodrigue De Schaetzen

Doctorat - UdeM

Doctorat - UdeM

Doctorat - UdeM

Doctorat - UdeM

Co-superviseur⋅e :

Glen Berseth

Anshul Gupta

Collaborateur·rice de recherche - UdeM

Co-superviseur⋅e :

Sarath Chandar

Site web

Kumaraditya Gupta

Collaborateur·rice de recherche

Co-superviseur⋅e :

Alihusein Kuwajerwala

Maîtrise recherche - UdeM

Site web

Zhen Liu

Doctorat - UdeM

Co-superviseur⋅e :

Doctorat - UdeM

Sacha Morin

Doctorat - UdeM

Superviseur⋅e principal⋅e :

Guy Wolf

Samer Nashed

Postdoctorat - UdeM

Yann Pequignot

Collaborateur·rice de recherche - Université Laval

miguel-angel.saavedra-ruiz@mila.quebec

Luke Rowe

Doctorat - UdeM

Co-superviseur⋅e :

Miguel Angel Saavedra Ruiz

Doctorat - UdeM

Visiteur de recherche indépendant - UdeM

Mahtab Sandhu

Maîtrise recherche - UdeM

Comment représenter efficacement le maillage non étanche de t-shirts?

Billets de blogue

Visuel de l'Article sur la représentation du maillage non étanche de t-shirts

15 mai 2024

par

Zhen Liu

Yao Feng

Yuliang Xiu

Weiyang Liu

Liam Paull

Michael J. Black

Bernhard Scholkopf

Lire l'article

Sample Efficient Deep Reinforcement Learning Via Uncertainty Estimation

9 mai 2022

Estimation d’incertitude pour un apprentissage par renforcement profond plus efficient

par

Vincent Mai

Kaustubh Mani

Liam Paull

Lire l'article

La-MAML: Look-ahead Meta-Learning for Continual Learning

19 novembre 2021

Méta-apprentissage prospectif pour l’apprentissage continu (La-MAML)

par

Gunshi Gupta

Liam Paull

Lire l'article

Publications

Rethinking Teacher-Student Curriculum Learning through the Cooperative Mechanics of Experience

Manfred Diaz

Andrea Tacchetti

Teacher-Student Curriculum Learning (TSCL) is a curriculum learning framework that draws inspiration from human cultural transmission and le… (voir plus)arning. It involves a teacher algorithm shaping the learning process of a learner algorithm by exposing it to controlled experiences. Despite its success, understanding the conditions under which TSCL is effective remains challenging. In this paper, we propose a data-centric perspective to analyze the underlying mechanics of the teacher-student interactions in TSCL. We leverage cooperative game theory to describe how the composition of the set of experiences presented by the teacher to the learner, as well as their order, influences the performance of the curriculum that is found by TSCL approaches. To do so, we demonstrate that for every TSCL problem, there exists an equivalent cooperative game, and several key components of the TSCL framework can be reinterpreted using game-theoretic principles. Through experiments covering supervised learning, reinforcement learning, and classical games, we estimate the cooperative values of experiences and use value-proportional curriculum mechanisms to construct curricula, even in cases where TSCL struggles. The framework and experimental setup we present in this work represent a novel foundation for a deeper exploration of TSCL, shedding light on its underlying mechanisms and providing insights into its broader applicability in machine learning.

2024-09-17

TMLR (accepté)

CtRL-Sim: Reactive and Controllable Driving Agents with Offline Reinforcement Learning

Luke Rowe

Roger Girgis

Anthony Gosselin

Bruno Carrez

Florian Golemo

Felix Heide

Chris Pal

Evaluating autonomous vehicle stacks (AVs) in simulation typically involves replaying driving logs from real-world recorded traffic. However… (voir plus), agents replayed from offline data do not react to the actions of the AV, and their behaviour cannot be easily controlled to simulate counterfactual scenarios. Existing approaches have attempted to address these shortcomings by proposing methods that rely on heuristics or learned generative models of real-world data but these approaches either lack realism or necessitate costly iterative sampling procedures to control the generated behaviours. In this work, we take an alternative approach and propose CtRL-Sim, a method that leverages return-conditioned offline reinforcement learning within a physics-enhanced Nocturne simulator to efficiently generate reactive and controllable traffic agents. Specifically, we process real-world driving data through the Nocturne simulator to generate a diverse offline reinforcement learning dataset, annotated with various reward terms. With this dataset, we train a return-conditioned multi-agent behaviour model that allows for fine-grained manipulation of agent behaviours by modifying the desired returns for the various reward components. This capability enables the generation of a wide range of driving behaviours beyond the scope of the initial dataset, including those representing adversarial behaviours. We demonstrate that CtRL-Sim can efficiently generate diverse and realistic safety-critical scenarios while providing fine-grained control over agent behaviours. Further, we show that fine-tuning our model on simulated safety-critical scenarios generated by our model enhances this controllability.

2024-09-05

robot-learning.org/CoRL/2024/Conference (accepté)

Milnor-Myerson Games and The Principles of Artificial Principal-Agent Problems

Manfred Diaz

Joel Z Leibo

In this paper, we introduce Milnor-Myerson games, a multiplayer interaction structure at the core of machine learning (ML), to shed light on… (voir plus) the fundamental principles and implications the artificial principal-agent problem has had in landmark ML results like AlphaGo and large language models (LLMs).

2024-06-04

rl-conference.cc/RLC/2024/Workshop/Finding_the_Frame (poster)

ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planning

Qiao Gu

Alihusein Kuwajerwala

Sacha Morin

Krishna Murthy

Bipasha Sen

Aditya Agarwal

Corban Rivera

William Paul

Kirsty Ellis

Rama Chellappa

Chuang Gan

Celso M de Melo

Joshua B. Tenenbaum

Antonio Torralba

Florian Shkurti

For robots to perform a wide variety of tasks, they require a 3D representation of the world that is semantically rich, yet compact and effi… (voir plus)cient for task-driven perception and planning. Recent approaches have attempted to leverage features from large vision-language models to encode semantics in 3D representations. However, these approaches tend to produce maps with per-point feature vectors, which do not scale well in larger environments, nor do they contain semantic spatial relationships between entities in the environment, which are useful for downstream planning. In this work, we propose ConceptGraphs, an open-vocabulary graph-structured representation for 3D scenes. ConceptGraphs is built by leveraging 2D foundation models and fusing their output to 3D by multi-view association. The resulting representations generalize to novel semantic classes, without the need to collect large 3D datasets or finetune models. We demonstrate the utility of this representation through a number of downstream planning tasks that are specified through abstract (language) prompts and require complex reasoning over spatial and semantic concepts. (Project page: https://concept-graphs.github.io/ Explainer video: https://youtu.be/mRhNkQwRYnc )

2024-05-13

2024 IEEE International Conference on Robotics and Automation (ICRA) (publié)

BACS: Background Aware Continual Semantic Segmentation

Mostafa ElAraby

Ali Harakeh

Semantic segmentation plays a crucial role in enabling comprehensive scene understanding for robotic systems. However, generating annotation… (voir plus)s is challenging, requiring labels for every pixel in an image. In scenarios like autonomous driving, there's a need to progressively incorporate new classes as the operating environment of the deployed agent becomes more complex. For enhanced annotation efficiency, ideally, only pixels belonging to new classes would be annotated. This approach is known as Continual Semantic Segmentation (CSS). Besides the common problem of classical catastrophic forgetting in the continual learning setting, CSS suffers from the inherent ambiguity of the background, a phenomenon we refer to as the"background shift'', since pixels labeled as background could correspond to future classes (forward background shift) or previous classes (backward background shift). As a result, continual learning approaches tend to fail. This paper proposes a Backward Background Shift Detector (BACS) to detect previously observed classes based on their distance in the latent space from the foreground centroids of previous steps. Moreover, we propose a modified version of the cross-entropy loss function, incorporating the BACS detector to down-weight background pixels associated with formerly observed classes. To combat catastrophic forgetting, we employ masked feature distillation alongside dark experience replay. Additionally, our approach includes a transformer decoder capable of adjusting to new classes without necessitating an additional classification head. We validate BACS's superior performance over existing state-of-the-art methods on standard CSS benchmarks.

2024-04-19

ArXiv (prépublication)

Correction to: Multi-agent reinforcement learning for fast-timescale demand response of residential loads

Vincent Mai

Philippe Maisonneuve

Tianyu Zhang

Hadi Nekoei

Antoine Lesage-Landry

2024-02-23

Machine-mediated learning (publié)

Ghost on the Shell: An Expressive Representation of General 3D Shapes

Zhen Liu

Yao Feng

Yuliang Xiu

Weiyang Liu

Michael J. Black

Bernhard Schölkopf

2024-01-16

ICLR.cc/2024/Conference (présentation orale)

GROOD: GRadient-aware Out-Of-Distribution detection in interpolated manifolds

Mostafa ElAraby

Sabyasachi Sahoo

Yann Batiste Pequignot

Paul Novello

2023-12-22

ArXiv (prépublication)

Rethinking Teacher-Student Curriculum Learning under the Cooperative Mechanics of Experience

Manfred Diaz

Andrea Tacchetti

Teacher-Student Curriculum Learning (TSCL) is a curriculum learning framework that draws inspiration from human cultural transmission and le… (voir plus)arning. It involves a teacher algorithm shaping the learning process of a learner algorithm by exposing it to controlled experiences. Despite its success, understanding the conditions under which TSCL is effective remains challenging. In this paper, we propose a data-centric perspective to analyze the underlying mechanics of the teacher-student interactions in TSCL. We leverage cooperative game theory to describe how the composition of the set of experiences presented by the teacher to the learner, as well as their order, influences the performance of the curriculum that are found by TSCL approaches. To do so, we demonstrate that for every TSCL problem, there exists an equivalent cooperative game, and several key components of the TSCL framework can be reinterpreted using game-theoretic principles. Through experiments covering supervised learning, reinforcement learning, and classical games, we estimate the cooperative values of experiences and use value-proportional curriculum mechanisms to construct curricula, even in cases where TSCL struggles. The framework and experimental setup we present in this work represent a foundation that can be used for a deeper exploration of TSCL, shedding light on its underlying mechanisms and providing insights into its broader applicability in machine learning.

2023-10-28

NeurIPS.cc/2023/Workshop/ALOE (poster)

Ghost on the Shell: An Expressive Representation of General 3D Shapes

Zhen Liu

Yao Feng

Yuliang Xiu

Weiyang Liu

Michael J. Black

Bernhard Schölkopf

2023-10-23

ArXiv (prépublication)

One-4-All: Neural Potential Fields for Embodied Navigation

Sacha Morin

Miguel Saavedra-Ruiz

A fundamental task in robotics is to navigate between two locations. In particular, real-world navigation can require long-horizon planning … (voir plus)using high-dimensional RGB images, which poses a substantial challenge for end-to-end learning-based approaches. Current semi-parametric methods instead achieve long-horizon navigation by combining learned modules with a topological memory of the environment, often represented as a graph over previously collected images. However, using these graphs in practice requires tuning a number of pruning heuristics. These heuristics are necessary to avoid spurious edges, limit runtime memory usage and maintain reasonably fast graph queries in large environments. In this work, we present One-4-All (O4A), a method leveraging self-supervised and manifold learning to obtain a graph-free, end-to-end navigation pipeline in which the goal is specified as an image. Navigation is achieved by greedily minimizing a potential function defined continuously over image embeddings. Our system is trained offline on non-expert exploration sequences of RGB data and controls, and does not require any depth or pose measurements. We show that 04A can reach long-range goals in 8 simulated Gibson indoor environments and that resulting embeddings are topologically similar to ground truth maps, even if no pose is observed. We further demonstrate successful real-world navigation using a Jackal UGV platform.aaProject page https://montrealrobotics.ca/o4a/.

2023-10-01

2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (publié)

Self-Supervised Image-to-Point Distillation via Semantically Tolerant Contrastive Loss

Anas Mahmoud

Jordan S. K. Hu

Tianshu Kuai

Ali Harakeh

Steven L. Waslander

An effective framework for learning 3D representations for perception tasks is distilling rich self-supervised image features via contrastiv… (voir plus)e learning. However, image-to-point representation learning for autonomous driving datasets faces two main challenges: 1) the abundance of self-similarity, which results in the contrastive losses pushing away semantically similar point and image regions and thus disturbing the local semantic structure of the learned representations, and 2) severe class imbalance as pretraining gets dominated by over-represented classes. We propose to alleviate the self-similarity problem through a novel semantically tolerant image-to-point contrastive loss that takes into consideration the semantic distance between positive and negative image regions to minimize contrasting semantically similar point and image regions. Additionally, we address class imbalance by designing a class-agnostic balanced loss that approximates the degree of class imbalance through an aggregate sample-to-samples semantic similarity measure. We demonstrate that our semantically-tolerant contrastive loss with class balancing improves state-of-the-art 2D-to-3D representation learning in all evaluation settings on 3D semantic segmentation. Our method consistently outperforms state-of-the-art 2D-to-3D representation learning frameworks across a wide range of 2D self-supervised pretrained models.

2023-06-17

2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (publié)