Gregory Dudek

Github

Google Scholar

Steve Wen

Maîtrise recherche - McGill

Superviseur⋅e principal⋅e :

Doina Precup

Publications

Multimodal dynamics modeling for off-road autonomous vehicles

Jean-François Tremblay

Travis Manderson

Aurélio Noca

Dynamics modeling in outdoor and unstructured environments is difficult because different elements in the environment interact with the robo… (voir plus)t in ways that can be hard to predict. Leveraging multiple sensors to perceive maximal information about the robot's environment is thus crucial when building a model to perform predictions about the robot's dynamics with the goal of doing motion planning. We design a model capable of long-horizon motion predictions, leveraging vision, lidar and proprioception, which is robust to arbitrarily missing modalities at test time. We demonstrate in simulation that our model is able to leverage vision to predict traction changes. We then test our model using a real-world challenging dataset of a robot navigating through a forest, performing predictions in trajectories unseen during training. We try different modality combinations at test time and show that, while our model performs best when all modalities are present, it is still able to perform better than the baseline even when receiving only raw vision input and no proprioception, as well as when only receiving proprioception. Overall, our study demonstrates the importance of leveraging multiple sensors when doing dynamics modeling in outdoor conditions.

2021-06-04

2021 IEEE International Conference on Robotics and Automation (ICRA) (publié)

Learning Intuitive Physics with Multimodal Generative Models

Sahand Rezaei-Shoshtari

Francois Hogan

M. Jenkin

Predicting the future interaction of objects when they come into contact with their environment is key for autonomous agents to take intelli… (voir plus)gent and anticipatory actions. This paper presents a perception framework that fuses visual and tactile feedback to make predictions about the expected motion of objects in dynamic scenes. Visual information captures object properties such as 3D shape and location, while tactile information provides critical cues about interaction forces and resulting object motion when it makes contact with the environment. Utilizing a novel See-Through-your-Skin (STS) sensor that provides high resolution multimodal sensing of contact surfaces, our system captures both the visual appearance and the tactile properties of objects. We interpret the dual stream signals from the sensor using a Multimodal Variational Autoencoder (MVAE), allowing us to capture both modalities of contacting objects and to develop a mapping from visual to tactile interaction and vice-versa. Additionally, the perceptual system can be used to infer the outcome of future physical interactions, which we validate through simulated and real-world experiments in which the resting state of an object is predicted from given initial conditions.

2021-05-17

Proceedings of the AAAI Conference on Artificial Intelligence (publié)

Seeing Through your Skin: Recognizing Objects with a Novel Visuotactile Sensor

Francois Hogan

M. Jenkin

Sahand Rezaei-Shoshtari

Yogesh Girdhar

We introduce a new class of vision-based sensor and associated algorithmic processes that combine visual imaging with high-resolution tactil… (voir plus)e sending, all in a uniform hardware and computational architecture. We demonstrate the sensor’s efficacy for both multi-modal object recognition and metrology. Object recognition is typically formulated as an unimodal task, but by combining two sensor modalities we show that we can achieve several significant performance improvements. This sensor, named the See-Through-your-Skin sensor (STS), is designed to provide rich multi-modal sensing of contact surfaces. Inspired by recent developments in optical tactile sensing technology, we address a key missing feature of these sensors: the ability to capture a visual perspective of the region beyond the contact surface. Whereas optical tactile sensors are typically opaque, we present a sensor with a semitransparent skin that has the dual capabilities of acting as a tactile sensor and/or as a visual camera depending on its internal lighting conditions. This paper details the design of the sensor, showcases its dual sensing capabilities, and presents a deep learning architecture that fuses vision and touch. We validate the ability of the sensor to classify household objects, recognize fine textures, and infer their physical properties both through numerical simulations and experiments with a smart countertop prototype.

2021-01-02

2021 IEEE Winter Conference on Applications of Computer Vision (WACV) (publié)

MBAIL: Multi-Batch Best Action Imitation Learning utilizing Sample Transfer and Policy Distillation

Dingwei Wu

Tianyu Li

M. Jenkin

Steve Liu

Batch reinforcement learning (RL) aims to learn a good control policy from a previously collected dataset without requiring additional inter… (voir plus)actions with the environment. Unfortunately, in the real world, we may only have a limited amount of training data for tasks we are interested in. Most batch RL methods are intended to learn a policy over one ﬁxed dataset, and are not intended to learn a policy that can perform well over other tasks. How can we leverage the advantages of batch RL while dealing with limited training data is another challenge in real world. In this work, we propose to add sample transfer and policy distillation to a leading Batch RL approach. The proposed methods are evaluated on multiple control tasks to showcase their effectiveness.

2020-12-31

(publié)

www.semanticscholar.org

Learning Domain Randomization Distributions for Training Robust Locomotion Policies

Melissa Mozian

Juan Camilo Gamboa Higuera

This paper considers the problem of learning behaviors in simulation without knowledge of the precise dynamical properties of the target rob… (voir plus)ot platform(s). In this context, our learning goal is to mutually maximize task efficacy on each environment considered and generalization across the widest possible range of environmental conditions. The physical parameters of the simulator are modified by a component of our technique that learns the Domain Randomization (DR) that is appropriate at each learning epoch to maximally challenge the current behavior policy, without being overly challenging, which can hinder learning progress. This so-called sweet spot distribution is a selection of simulated domains with the following properties: 1) The trained policy should be successful in environments sampled from the domain randomization distribution; and 2) The DR distribution made as wide as possible, to increase variability in the environments. These properties aim to ensure the trajectories encountered in the target system are close to those observed during training, as existing methods in machine learning are better suited for interpolation than extrapolation. We show how adapting the DR distribution while training context-conditioned policies results in improvements on jump-start and asymptotic performance when transferring a learned policy to the target environment1.

2020-10-23

2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (publié)

PresSense: Passive Respiration Sensing via Ambient WiFi Signals in Noisy Environments

Yi Tian Xu

X. T. Chen

Xue Liu

Passive sensing with ambient WiFi signals is a promising technique that will enable new types of human-robot interactions while preserving u… (voir plus)sers' privacy. Here, we present PresSense, a system for human respiration sensing in noisy environments. Unlike existing WiFi-based respiration sensors, we employ a human presence detector, improving the robustness in scenarios where no human is present in an Area Of Interest (AOI). We also integrate our novel feature, Peak Distance Histogram (PDH), with other classic WiFi features to achieve better accuracy when someone is present in the AOI. We tested our system using commodity WiFi devices in an office room. Our PresSense outperforms the state of the arts in both respiration rate estimation and presence detection.

2020-10-23

IEEE/RJS International Conference on Intelligent RObots and Systems (publié)

Learning to Drive Off Road on Smooth Terrain in Unstructured Environments Using an On-Board Camera and Sparse Aerial Images

Travis Manderson

Stefan Wapnick

We present a method for learning to drive on smooth terrain while simultaneously avoiding collisions in challenging off-road and unstructure… (voir plus)d outdoor environments using only visual inputs. Our approach applies a hybrid model-based and model-free reinforcement learning method that is entirely self-supervised in labeling terrain roughness and collisions using on-board sensors. Notably, we provide both first-person and overhead aerial image inputs to our model. We find that the fusion of these complementary inputs improves planning foresight and makes the model robust to visual obstructions. Our results show the ability to generalize to environments with plentiful vegetation, various types of rock, and sandy trails. During evaluation, our policy attained 90% smooth terrain traversal and reduced the proportion of rough terrain driven over by 6.1 times compared to a model using only first-person imagery.

2020-08-30

2020 IEEE International Conference on Robotics and Automation (ICRA) (publié)

Dynamic planning of redundant robots within a set-based task-priority inverse kinematics framework.

Daniele Di Vito

Mathieux Bergeron

Gianluca Antonelli

This work presents the dynamic planning of redundant robots by merging a global and local planner. The global planner is implemented as a sa… (voir plus)mpling-based algorithm which works in the reduced-dimensionality of the robot workspace applying the Cartesian constraints only. The output trajectory is then checked within a framework of set-based task priority inverse kinematics verifying the fulfillment of the other task constraints. The inverse kinematics framework is used also in real-time as local motion control to ensure a reactive behaviour to address, e.g., mismatch between the apriori information and on-line perception acquisition. During the movement, the motion planner runs in background to adapt to changes in the environment or, in general, to continuously optimize the path. The proposed method is experimentally validated with a Kinova Jaco2 7 degrees of freedom manipulator.

2020-07-31

Conference on Control Technology and Applications (publié)

Vision-Based Goal-Conditioned Policies for Underwater Navigation in the Presence of Obstacles.

Travis Manderson

Juan Camilo Gamboa Higuera

Stefan Wapnick

Jean-François Tremblay

Florian Shkurti

We present Nav2Goal, a data-efficient and end-to-end learning method for goal-conditioned visual navigation. Our technique is used to train … (voir plus)a navigation policy that enables a robot to navigate close to sparse geographic waypoints provided by a user without any prior map, all while avoiding obstacles and choosing paths that cover user-informed regions of interest. Our approach is based on recent advances in conditional imitation learning. General-purpose, safe and informative actions are demonstrated by a human expert. The learned policy is subsequently extended to be goal-conditioned by training with hindsight relabelling, guided by the robot's relative localization system, which requires no additional manual annotation. We deployed our method on an underwater vehicle in the open ocean to collect scientifically relevant data of coral reefs, which allowed our robot to operate safely and autonomously, even at very close proximity to the coral. Our field deployments have demonstrated over a kilometer of autonomous visual navigation, where the robot reaches on the order of 40 waypoints, while collecting scientifically relevant data. This is done while travelling within 0.5 m altitude from sensitive corals and exhibiting significant learned agility to overcome turbulent ocean conditions and to actively avoid collisions.

2020-07-11

Robotics: Science and Systems XVI (publié)

Depth Prediction for Monocular Direct Visual Odometry

Ran Cheng

Christopher Agia

Depth prediction from monocular images with deep CNNs is a topic of increasing interest to the community. Advances have lead to models capab… (voir plus)le of predicting disparity maps with consistent scale, which are an acceptable prior for gradient-based direct methods. With this in consideration, we exploit depth prediction as a candidate prior for the coarse initialization, tracking, and marginalization steps of the direct visual odometry system, enabling the second-order optimizer to converge faster into a precise global minimum. In addition, the given depth prior supports large baseline stereo scenarios, maintaining robust pose estimations against challenging motion states such as in-place rotation. We further refine our pose estimation with semi-online loop closure. The experiments on KITTI demonstrate that our proposed method achieves state- of-the-art performance compared to both traditional direct visual odometry and learning-based counterparts.

2020-05-12

2020 17th Conference on Computer and Robot Vision (CRV) (publié)

Navigation in the Service of Enhanced Pose Estimation

Travis Manderson

Ran Cheng

2020-01-22

Springer Proceedings in Advanced Robotics (publié)

Seeing Through Your Skin: A Novel Visuo-Tactile Sensor for Robotic Manipulation

Francois Hogan

Sahand Rezaei-Shoshtari

M. Jenkin

Yashveer Girdhar