Portrait de Liam Paull

Liam Paull

Membre académique principal
paulll@mila.quebec
Chaire en IA Canada-CIFAR
Professeur adjoint, Université de Montréal, Département d'informatique et de recherche opérationnelle
Sujets de recherche
Apprentissage profond
Robotique
Vision par ordinateur
Site web
Google Scholar
LinkedIn
X

Biographie

Liam Paull est professeur adjoint à l'Université de Montréal et codirige le Laboratoire de robotique et d’IA intégrative de Montréal (REAL). Son laboratoire se concentre sur les problèmes de robotique, y compris la construction de représentations du monde (pour la localisation et la cartographie simultanées, par exemple), la modélisation de l'incertitude et la construction de meilleurs flux de travail pour enseigner de nouvelles tâches aux agents robotiques (notamment par la simulation ou la démonstration). Auparavant, Liam Paull a été chercheur au Computer Science and Artificial Intelligence Laboratory (CSAIL) du Massachusetts Institute of Technology (MIT), où il a dirigé le projet de voiture autonome financé par le Toyota Research Institute (TRI). Il a également été chercheur postdoctoral au laboratoire de robotique marine du MIT, où il a travaillé sur la technique SLAM (Simultaneous Localization and Mapping) pour les robots sous-marins. Il a obtenu son doctorat en 2013 à l'Université du Nouveau-Brunswick : il s’y est intéressé à la planification robuste et adaptative pour les véhicules sous-marins. Il est cofondateur et directeur de la Fondation Duckietown, dont l'objectif est de rendre accessibles à tous·tes les expériences d'apprentissage de la robotique.

Étudiants actuels

Publications

f-Cal: Aleatoric uncertainty quantification for robot perception via calibrated neural regression
Dhaivat Bhatt
Kaustubh Mani
Dishank Bansal
Krishna Murthy
Hanju Lee
Liam Paull
While modern deep neural networks are performant perception modules, performance (accuracy) alone is insufficient, particularly for safety-c… (voir plus)ritical robotic applications such as self-driving vehicles. Robot autonomy stacks also require these otherwise blackbox models to produce reliable and calibrated measures of confidence on their predictions. Existing approaches estimate uncertainty from these neural network perception stacks by modifying network architectures, inference procedure, or loss functions. However, in general, these methods lack calibration, meaning that the predictive uncertainties do not faithfully represent the true underlying uncertainties (process noise). Our key insight is that calibration is only achieved by imposing constraints across multiple examples, such as those in a mini-batch; as opposed to existing approaches which only impose constraints per-sample, often leading to overconfident (thus miscalibrated) uncertainty estimates. By enforcing the distribution of outputs of a neural network to resemble a target distribution by minimizing an
2022-05-22
IEEE Conference on Robotics and Automation (publié)
doi.org
Sample Efficient Deep Reinforcement Learning via Uncertainty Estimation
Vincent Mai
Kaustubh Mani
Liam Paull
In model-free deep reinforcement learning (RL) algorithms, using noisy value estimates to supervise policy evaluation and optimization is de… (voir plus)trimental to the sample efficiency. As this noise is heteroscedastic, its effects can be mitigated using uncertainty-based weights in the optimization process. Previous methods rely on sampled ensembles, which do not capture all aspects of uncertainty. We provide a systematic analysis of the sources of uncertainty in the noisy supervision that occurs in RL, and introduce inverse-variance RL, a Bayesian framework which combines probabilistic ensembles and Batch Inverse Variance weighting. We propose a method whereby two complementary uncertainty estimation methods account for both the Q-value and the environment stochasticity to better mitigate the negative impacts of noisy supervision. Our results show significant improvement in terms of sample efficiency on discrete and continuous control tasks.
2022-04-24
International Conference on Learning Representations (Accept (Spotlight))
doi.org
openreview.net
Iterative Teaching by Label Synthesis
Weiyang Liu
Zhen Liu
Hanchen Wang
Liam Paull
Bernhard Schölkopf
Adrian Weller
In this paper, we consider the problem of iterative machine teaching, where a teacher provides examples sequentially based on the current it… (voir plus)erative learner. In contrast to previous methods that have to scan over the entire pool and select teaching examples from it in each iteration, we propose a label synthesis teaching framework where the teacher randomly selects input teaching examples (e.g., images) and then synthesizes suitable outputs (e.g., labels) for them. We show that this framework can avoid costly example selection while still provably achieving exponential teachability. We propose multiple novel teaching algorithms in this framework. Finally, we empirically demonstrate the value of our framework.
2021-12-06
NEURIPS (Accept (Spotlight))
doi.org
openreview.net
Your GAN is Secretly an Energy-based Model and You Should Use Discriminator Driven Latent Sampling
Tong Che
Ruixiang Zhang
Jascha Sohl-Dickstein
Hugo Larochelle
Liam Paull
Yuan Cao
Yoshua Bengio
We show that the sum of the implicit generator log-density …
2020-12-06
Conference on Neural Information Processing Systems (Accept (poster))
doi.org
arxiv.org
Perceptual Generative Autoencoders
Zijun Zhang
Ruixiang Zhang
Zongpeng Li
Yoshua Bengio
Liam Paull
Modern generative models are usually designed to match target distributions directly in the data space, where the intrinsic dimension of dat… (voir plus)a can be much lower than the ambient dimension. We argue that this discrepancy may contribute to the difficulties in training generative models. We therefore propose to map both the generated and target distributions to a latent space using the encoder of a standard autoencoder, and train the generator (or decoder) to match the target distribution in the latent space. Specifically, we enforce the consistency in both the data space and the latent space with theoretically justified data and latent reconstruction losses. The resulting generative model, which we call a perceptual generative autoencoder (PGA), is then trained with a maximum likelihood or variational autoencoder (VAE) objective. With maximum likelihood, PGAs generalize the idea of reversible generative models to unrestricted neural network architectures and arbitrary number of latent dimensions. When combined with VAEs, PGAs substantially improve over the baseline VAEs in terms of sample quality. Compared to other autoencoder-based generative models using simple priors, PGAs achieve state-of-the-art FID scores on CIFAR-10 and CelebA.
2020-11-20
Proceedings of the 37th International Conference on Machine Learning (publié)
doi.org
proceedings.mlr.press
Active Domain Randomization
Bhairav Mehta
Manfred Diaz
Florian Golemo
Christopher Pal
Liam Paull
Domain randomization is a popular technique for improving domain transfer, often used in a zero-shot setting when the target domain is unkno… (voir plus)wn or cannot easily be used for training. In this work, we empirically examine the effects of domain randomization on agent generalization. Our experiments show that domain randomization may lead to suboptimal, high-variance policies, which we attribute to the uniform sampling of environment parameters. We propose Active Domain Randomization, a novel algorithm that learns a parameter sampling strategy. Our method looks for the most informative environment variations within the given randomization ranges by leveraging the discrepancies of policy rollouts in randomized and reference environment instances. We find that training more frequently on these instances leads to better overall agent generalization. In addition, when domain randomization and policy transfer fail, Active Domain Randomization offers more insight into the deficiencies of both the chosen parameter ranges and the learned policy, allowing for more focused debugging. Our experiments across various physics-based simulated and a real-robot task show that this enhancement leads to more robust, consistent policies.
2020-05-11
Proceedings of the Conference on Robot Learning (publié)
proceedings.mlr.press
proceedings.mlr.press
Curriculum in Gradient-Based Meta-Reinforcement Learning
Bhairav Mehta
Tristan Deleu
Sharath Chandra Raparthy
Christopher Pal
Liam Paull
Gradient-based meta-learners such as Model-Agnostic Meta-Learning (MAML) have shown strong few-shot performance in supervised and reinforcem… (voir plus)ent learning settings. However, specifically in the case of meta-reinforcement learning (meta-RL), we can show that gradient-based meta-learners are sensitive to task distributions. With the wrong curriculum, agents suffer the effects of meta-overfitting, shallow adaptation, and adaptation instability. In this work, we begin by highlighting intriguing failure cases of gradient-based meta-RL and show that task distributions can wildly affect algorithmic outputs, stability, and performance. To address this problem, we leverage insights from recent literature on domain randomization and propose meta Active Domain Randomization (meta-ADR), which learns a curriculum of tasks for gradient-based meta-RL in a similar as ADR does for sim2real transfer. We show that this approach induces more stable policies on a variety of simulated locomotion and navigation tasks. We assess in- and out-of-distribution generalization and find that the learned task distributions, even in an unstructured task space, greatly improve the adaptation performance of MAML. Finally, we motivate the need for better benchmarking in meta-RL that prioritizes \textit{generalization} over single-task adaption performance.
2020-02-18
ArXiv (prépublication)
arxiv.org
arxiv.org
A Data-Efficient Framework for Training and Sim-to-Real Transfer of Navigation Policies
Homanga Bharadhwaj
Zihan Wang
Yoshua Bengio
Liam Paull
Learning effective visuomotor policies for robots purely from data is challenging, but also appealing since a learning-based system should n… (voir plus)ot require manual tuning or calibration. In the case of a robot operating in a real environment the training process can be costly, time-consuming, and even dangerous since failures are common at the start of training. For this reason, it is desirable to be able to leverage \textit{simulation} and \textit{off-policy} data to the extent possible to train the robot. In this work, we introduce a robust framework that plans in simulation and transfers well to the real environment. Our model incorporates a gradient-descent based planning module, which, given the initial image and goal image, encodes the images to a lower dimensional latent state and plans a trajectory to reach the goal. The model, consisting of the encoder and planner modules, is trained through a meta-learning strategy in simulation first. We subsequently perform adversarial domain transfer on the encoder by using a bank of unlabelled but random images from the simulation and real environments to enable the encoder to map images from the real and simulated environments to a similarly distributed latent representation. By fine tuning the entire model (encoder + planner) with far fewer real world expert demonstrations, we show successful planning performances in different navigation tasks.
2019-05-19
2019 International Conference on Robotics and Automation (ICRA) (publié)
doi.org
arxiv.org