David Meger

William Bonilla

PhD - McGill University

Valliappan Chidambaram Adaikkappan

PhD - McGill University

Wesley Chung

PhD - McGill University

Co-supervisor :

Farnoosh Faraji

PhD - McGill University

Co-supervisor :

PhD - McGill University

Principal supervisor :

Anas Houssaini

Master's Research - McGill University

Co-supervisor :

Hsiu-Chin Lin

Zina Kamel

Master's Research - McGill University

Co-supervisor :

Hsiu-Chin Lin

Melissa Mozifian

PhD - McGill University

Principal supervisor :

Joelle Pineau

Louis Petit

Postdoctorate - McGill University

Website

Sahand Rezaei-Shoshtari

PhD - McGill University

Principal supervisor :

PhD - McGill University

Jean-François Tremblay

PhD - McGill University

Steven Wang

Master's Research - McGill University

Harley Wiltzer

PhD - McGill University

Co-supervisor :

Marc Gendron-Bellemare

Master's Research - McGill University

Publications

Topological mapping for traversability-aware long-range navigation in off-road terrain

Jean-Franccois Tremblay

Julie Alhosh

Louis Petit

Faraz Lotfi

Lara Landauro

Autonomous robots navigating in off-road terrain like forests open new opportunities for automation. While off-road navigation has been stud… (see more)ied, existing work often relies on clearly delineated pathways. We present a method allowing for long-range planning, exploration and low-level control in unknown off-trail forest terrain, using vision and GPS only. We represent outdoor terrain with a topological map, which is a set of panoramic snapshots connected with edges containing traversability information. A novel traversability analysis method is demonstrated, predicting the existence of a safe path towards a target in an image. Navigating between nodes is done using goal-conditioned behavior cloning, leveraging the power of a pretrained vision transformer. An exploration planner is presented, efficiently covering an unknown off-road area with unknown traversability using a frontiers-based approach. The approach is successfully deployed to autonomously explore two 400 meters squared forest sites unseen during training, in difficult conditions for navigation.

2024-10-02

ArXiv (preprint)

Action Gaps and Advantages in Continuous-Time Distributional Reinforcement Learning

Harley Wiltzer

Marc Gendron-Bellemare

Patrick Shafto

Yash Jhaveri

2024-09-25

NeurIPS.cc/2024/Conference (poster)

Parseval Regularization for Continual Reinforcement Learning

Wesley Chung

Lynn Cherif

2024-09-25

NeurIPS.cc/2024/Conference (poster)

Shedding Light on Large Generative Networks: Estimating Epistemic Uncertainty in Diffusion Models

Lucas Berry

Axel Brando

Generative diffusion models, notable for their large parameter count (exceeding 100 million) and operation within high-dimensional image spa… (see more)ces, pose significant challenges for traditional uncertainty estimation methods due to computational demands. In this work, we introduce an innovative framework, Diffusion Ensembles for Capturing Uncertainty (DECU), designed for estimating epistemic uncertainty for diffusion models. The DECU framework introduces a novel method that efficiently trains ensembles of conditional diffusion models by incorporating a static set of pre-trained parameters, drastically reducing the computational burden and the number of parameters that require training. Additionally, DECU employs Pairwise-Distance Estimators (PaiDEs) to accurately measure epistemic uncertainty by evaluating the mutual information between model outputs and weights in high-dimensional spaces. The effectiveness of this framework is demonstrated through experiments on the ImageNet dataset, highlighting its capability to capture epistemic uncertainty, specifically in under-sampled image classes.

2024-09-12

Proceedings of the Fortieth Conference on Uncertainty in Artificial Intelligence (published)

Imitation Learning from Observation through Optimal Transport

Wei-Di Chang

Scott Fujimoto

2024-05-14

rl-conference.cc/RLC/2024/Conference (published)

Constrained Robotic Navigation on Preferred Terrains Using LLMs and Speech Instruction: Exploiting the Power of Adverbs

Faraz Lotfi

Farnoosh Faraji

Nikhil Kakodkar

Travis Manderson

2024-04-02

ArXiv (preprint)

Policy Gradient Methods in the Presence of Symmetries and State Abstractions

Prakash Panangaden

Sahand Rezaei-Shoshtari

Rosie Zhao

Uncertainty-aware hybrid paradigm of nonlinear MPC and model-based RL for offroad navigation: Exploration of transformers in the predictive model

Faraz Lotfi

Khalil Virji

Farnoosh Faraji

Lucas Berry

Andrew Holliday

In this paper, we investigate a hybrid scheme that combines nonlinear model predictive control (MPC) and model-based reinforcement learning … (see more)(RL) for navigation planning of an autonomous model car across offroad, unstructured terrains without relying on predefined maps. Our innovative approach takes inspiration from BADGR, an LSTM-based network that primarily concentrates on environment modeling, but distinguishes itself by substituting LSTM modules with transformers to greatly elevate the performance our model. Addressing uncertainty within the system, we train an ensemble of predictive models and estimate the mutual information between model weights and outputs, facilitating dynamic horizon planning through the introduction of variable speeds. Further enhancing our methodology, we incorporate a nonlinear MPC controller that accounts for the intricacies of the vehicle's model and states. The model-based RL facet produces steering angles and quantifies inherent uncertainty. At the same time, the nonlinear MPC suggests optimal throttle settings, striking a balance between goal attainment speed and managing model uncertainty influenced by velocity. In the conducted studies, our approach excels over the existing baseline by consistently achieving higher metric values in predicting future events and seamlessly integrating the vehicle's kinematic model for enhanced decision-making. The code and the evaluation data are available at https://github.com/FARAZLOTFI/offroad_autonomous_navigation/).

2024-01-01

ICRA (published)

Generalizable Imitation Learning Through Pre-Trained Representations

Wei-Di Chang

Francois Hogan

In this paper we leverage self-supervised vision transformer models and their emergent semantic abilities to improve the generalization abil… (see more)ities of imitation learning policies. We introduce BC-ViT, an imitation learning algorithm that leverages rich DINO pre-trained Visual Transformer (ViT) patch-level embeddings to obtain better generalization when learning through demonstrations. Our learner sees the world by clustering appearance features into semantic concepts, forming stable keypoints that generalize across a wide range of appearance variations and object types. We show that this representation enables generalized behaviour by evaluating imitation learning across a diverse dataset of object manipulation tasks. Our method, data and evaluation approach are made available to facilitate further study of generalization in Imitation Learners.

2023-11-15

ArXiv (preprint)

For SALE: State-Action Representation Learning for Deep Reinforcement Learning

Scott Fujimoto

Wei-Di Chang

Edward J. Smith

Shixiang Shane Gu

In the field of reinforcement learning (RL), representation learning is a proven tool for complex image-based tasks, but is often overlooked… (see more) for environments with low-level states, such as physical control problems. This paper introduces SALE, a novel approach for learning embeddings that model the nuanced interaction between state and action, enabling effective representation learning from low-level states. We extensively study the design space of these embeddings and highlight important design considerations. We integrate SALE and an adaptation of checkpoints for RL into TD3 to form the TD7 algorithm, which significantly outperforms existing continuous control algorithms. On OpenAI gym benchmark tasks, TD7 has an average performance gain of 276.7% and 50.7% over TD3 at 300k and 5M time steps, respectively, and works in both the online and offline settings.

Leveraging World Model Disentanglement in Value-Based Multi-Agent Reinforcement Learning

Zhizun Wang

In this paper, we propose a novel model-based multi-agent reinforcement learning approach named Value Decomposition Framework with Disentang… (see more)led World Model to address the challenge of achieving a common goal of multiple agents interacting in the same environment with reduced sample complexity. Due to scalability and non-stationarity problems posed by multi-agent systems, model-free methods rely on a considerable number of samples for training. In contrast, we use a modularized world model, composed of action-conditioned, action-free, and static branches, to unravel the environment dynamics and produce imagined outcomes based on past experience, without sampling directly from the real environment. We employ variational auto-encoders and variational graph auto-encoders to learn the latent representations for the world model, which is merged with a value-based framework to predict the joint action-value function and optimize the overall training objective. We present experimental results in Easy, Hard, and Super-Hard StarCraft II micro-management challenges to demonstrate that our method achieves high sample efficiency and exhibits superior performance in defeating the enemy armies compared to other baselines.

2023-09-08

ArXiv (preprint)

Efficient Epistemic Uncertainty Estimation in Regression Ensemble Models Using Pairwise-Distance Estimators

Lucas Berry

This work introduces an efficient novel approach for epistemic uncertainty estimation for ensemble models for regression tasks using pairwis… (see more)e-distance estimators (PaiDEs). Utilizing the pairwise-distance between model components, these estimators establish bounds on entropy. We leverage this capability to enhance the performance of Bayesian Active Learning by Disagreement (BALD). Notably, unlike sample-based Monte Carlo estimators, PaiDEs exhibit a remarkable capability to estimate epistemic uncertainty at speeds up to 100 times faster while covering a significantly larger number of inputs at once and demonstrating superior performance in higher dimensions. To validate our approach, we conducted a varied series of regression experiments on commonly used benchmarks: 1D sinusoidal data,

2023-08-25

ArXiv (preprint)