Portrait of Liam Paull

Liam Paull

Core Academic Member
paulll@mila.quebec
Canada CIFAR AI Chair
Assistant Professor, Université de Montréal, Department of Computer Science and Operations Research
Website
Google Scholar
LinkedIn
X

Biography

Liam Paull is an associate professor at Université de Montréal and co-leads the Montréal Robotics and Embodied AI Lab (REAL). His lab focuses on a variety of robotics problems, including building representations of the world for such applications as simultaneous localization and mapping, modelling uncertainty, and building better workflows to teach robotic agents new tasks through, for example, simulation or demonstration.

Previously, Paull was a research scientist in the Computer Science and Artificial Intelligence Laboratory (CSAIL) at the Massachusetts Institute of Technology (MIT), where he led the autonomous car project funded by the Toyota Research Institute (TRI). He completed a postdoc with the Marine Robotics Group at MIT, where he worked on Simultaneous Localization and Mapping (SLAM) for underwater robots.

His PhD from the University of New Brunswick in 2013 focused on robust and adaptive planning for underwater vehicles. He is also the co-founder and director of the Duckietown Foundation, which is dedicated to making engaging robotics learning experiences accessible to everyone.

Current Students

Adam Burhan
Research Intern - Université de Montréal
adam.burhan@mila.quebec
Github
Alihusein Kuwajerwala
PhD - Université de Montréal
alihusein.kuwajerwala@mila.quebec
Website
Github
Google Scholar
Andrea Daniele
Postdoctorate - Université de Montréal
andreafrancesco.daniele@mila.quebec
Github
Charlie Gauthier
PhD - Université de Montréal
Co-supervisor :
Glen Berseth
charlie.gauthier@mila.quebec
Kaustubh Mani
PhD - Université de Montréal
kaustubh.mani@mila.quebec
Github
Google Scholar
Luke Rowe
PhD - Université de Montréal
Co-supervisor :
Chris Pal
luke.rowe@mila.quebec
Github
Google Scholar
Mahtab Sandhu
Master's Research - Université de Montréal
mahtab.sandhu@mila.quebec
Github
Google Scholar
Manfred Diaz Cabrera
PhD - Université de Montréal
diazcabm@mila.quebec
Website
Github
Google Scholar
Miguel Angel Saavedra Ruiz
PhD - Université de Montréal
miguel-angel.saavedra-ruiz@mila.quebec
Website
Github
Google Scholar
Moustafa Elarabi
PhD - Université de Montréal
elarabim@mila.quebec
Website
Github
Google Scholar
Ria Arora
Master's Research - Université de Montréal
Principal supervisor :
Guy Wolf
ria.arora@mila.quebec
Github
Ruixiang Zhang
PhD - Université de Montréal
Co-supervisor :
Yoshua Bengio
zhangrui@mila.quebec
Website
Sacha Morin
PhD - Université de Montréal
Principal supervisor :
Guy Wolf
sacha.morin@mila.quebec
Samer Nashed
Postdoctorate - Université de Montréal
samer.nashed@mila.quebec
Yann Pequignot
Collaborating researcher - Université Laval
yann.pequignot@mila.quebec
Zhen Liu
PhD - Université de Montréal
Co-supervisor :
Yoshua Bengio
liuzhen@mila.quebec

Publications

MeshDiffusion: Score-based Generative 3D Mesh Modeling
Zhen Liu
Yao Feng
Michael J. Black
Derek Nowrouzezahrai
Liam Paull
Weiyang Liu
We consider the task of generating realistic 3D shapes, which is useful for a variety of applications such as automatic scene generation and… (see more) physical simulation. Compared to other 3D representations like voxels and point clouds, meshes are more desirable in practice, because (1) they enable easy and arbitrary manipulation of shapes for relighting and simulation, and (2) they can fully leverage the power of modern graphics pipelines which are mostly optimized for meshes. Previous scalable methods for generating meshes typically rely on sub-optimal post-processing, and they tend to produce overly-smooth or noisy surfaces without fine-grained geometric details. To overcome these shortcomings, we take advantage of the graph structure of meshes and use a simple yet very effective generative modeling method to generate 3D meshes. Specifically, we represent meshes with deformable tetrahedral grids, and then train a diffusion model on this direct parameterization. We demonstrate the effectiveness of our model on multiple generative tasks.
2023-02-01
ICLR.cc/2023/Conference (notable)
doi.org
openreview.net
Robust and Controllable Object-Centric Learning through Energy-based Models
Ruixiang ZHANG
Tong Che
Boris Ivanovic
Renhao Wang
Marco Pavone
Yoshua Bengio
Liam Paull
Humans are remarkably good at understanding and reasoning about complex visual scenes. The capability of decomposing low-level observations … (see more)into discrete objects allows us to build a grounded abstract representation and identify the compositional structure of the world. Thus it is a crucial step for machine learning models to be capable of inferring objects and their properties from visual scene without explicit supervision. However, existing works on object-centric representation learning are either relying on tailor-made neural network modules or assuming sophisticated models of underlying generative and inference processes. In this work, we present EGO, a conceptually simple and general approach to learning object-centric representation through energy-based model. By forming a permutation-invariant energy function using vanilla attention blocks that are readily available in Transformers, we can infer object-centric latent variables via gradient-based MCMC methods where permutation equivariance is automatically guaranteed. We show that EGO can be easily integrated into existing architectures, and can effectively extract high-quality object-centric representations, leading to better segmentation accuracy and competitive downstream task performance. We empirically evaluate the robustness of the learned representation from EGO against distribution shift. Finally, we demonstrate the effectiveness of EGO in systematic compositional generalization, by recomposing learned energy functions for novel scene generation and manipulation.
2023-02-01
ICLR.cc/2023/Conference (poster)
doi.org
openreview.net
Multi-Agent Reinforcement Learning for Fast-Timescale Demand Response of Residential Loads
Vincent Mai
Philippe Maisonneuve
Tianyu Zhang
Hadi Nekoei
Liam Paull
Antoine Lesage-Landry
To integrate high amounts of renewable energy resources, electrical power grids must be able to cope with high amplitude, fast timescale var… (see more)iations in power generation. Frequency regulation through demand response has the potential to coordinate temporally flexible loads, such as air conditioners, to counteract these variations. Existing approaches for discrete control with dynamic constraints struggle to provide satisfactory performance for fast timescale action selection with hundreds of agents. We propose a decentralized agent trained with multi-agent proximal policy optimization with localized communication. We explore two communication frameworks: hand-engineered, or learned through targeted multi-agent communication. The resulting policies perform well and robustly for frequency regulation, and scale seamlessly to arbitrary numbers of houses for constant processing times.
2023-01-01
AAMAS (published)
doi.org
arxiv.org
Lifelong Topological Visual Navigation
Rey Reza Wiyatno
Anqi Xu
Liam Paull
Commonly, learning-based topological navigation approaches produce a local policy while preserving some loose connectivity of the space thro… (see more)ugh a topological map. Nevertheless, spurious or missing edges in the topological graph often lead to navigation failure. In this work, we propose a sampling-based graph building method, which results in sparser graphs yet with higher navigation performance compared to baseline methods. We also propose graph maintenance strategies that eliminate spurious edges and expand the graph as needed, which improves lifelong navigation performance. Unlike controllers that learn from fixed training environments, we show that our model can be fine-tuned using only a small number of collected trajectory images from a real-world environment where the agent is deployed. We demonstrate successful navigation after fine-tuning on real-world environments, and notably show significant navigation improvements over time by applying our lifelong graph maintenance strategies.
2022-10-01
IEEE Robotics and Automation Letters (published)
doi.org
arxiv.org
Monocular Robot Navigation with Self-Supervised Pretrained Vision Transformers
Miguel Saavedra-Ruiz
Sacha Morin
Liam Paull
In this work, we consider the problem of learning a perception model for monocular robot navigation using few annotated images. Using a Visi… (see more)on Transformer (ViT) pretrained with a label-free self-supervised method, we successfully train a coarse image segmentation model for the Duckietown environment using 70 training images. Our model performs coarse image segmentation at the
2022-06-02
2022 19th Conference on Robots and Vision (CRV) (published)
doi.org
arxiv.org
Monocular Robot Navigation with Self-Supervised Pretrained Vision Transformers
Miguel Saavedra-Ruiz
Sacha Morin
Liam Paull
In this work, we consider the problem of learning a perception model for monocular robot navigation using few annotated images. Using a Visi… (see more)on Transformer (ViT) pretrained with a label-free self-supervised method, we successfully train a coarse image segmentation model for the Duckietown environment using 70 training images. Our model performs coarse image segmentation at the
2022-03-07
ArXiv (preprint)
doi.org
arxiv.org
Lifelong Topological Visual Navigation
Rey Reza Wiyatno
Anqi Xu
Liam Paull
Commonly, learning-based topological navigation approaches produce a local policy while preserving some loose connectivity of the space thro… (see more)ugh a topological map. Nevertheless, spurious or missing edges in the topological graph often lead to navigation failure. In this work, we propose a sampling-based graph building method, which results in sparser graphs yet with higher navigation performance compared to baseline methods. We also propose graph maintenance strategies that eliminate spurious edges and expand the graph as needed, which improves lifelong navigation performance. Unlike controllers that learn from fixed training environments, we show that our model can be fine-tuned using only a small number of collected trajectory images from a real-world environment where the agent is deployed. We demonstrate successful navigation after fine-tuning on real-world environments, and notably show significant navigation improvements over time by applying our lifelong graph maintenance strategies.
2021-10-16
ArXiv (preprint)
doi.org
arxiv.org