Robotics

Robots are used worldwide in many industrial processes, and are getting better at helping humans every year. Machine learning algorithms are enhancing the capabilities of traditional robotics, and have become essential in making robots more adaptable to challenging situations.

People watch a robotic arm at work in a factory.

Embodied machine learning seeks to emulate the ways in which humans process information.  By using a wide variety of sensors on robotic hardware, researchers are able to help robots perceive, analyze, interact, and navigate through unpredictable physical environments. Mila researchers are tackling challenges such as better long-term planning for the use of robots in daily life, building representations of the world — including simultaneous localization and mapping — while creating better workflows to teach robotic agents new tasks.

Mila’s work also includes designing experimental machine learning algorithms to help robots perform better in industrial applications such as assembly and disassembly, meal preparation, and warehouse management.

Featured Projects

Engineers working with medical robotic equipment.

DROID

DROID is an initiative that aims to address the scarcity of comprehensive datasets in robotics, enhancing the development of manipulation algorithms for real-world applications.

Geometric shapes on a dark blue background.

ConceptGraphs

ConceptGraphs is a mapping system that builds 3D scene-graphs of objects and their relationships, enabling robots to perform complex navigation and object manipulation tasks.

Photo of Glen Berseth

AI can help us make robots more adaptable to unpredictable environments, which will lead to true robotics assistants in the real world. 

Glen Berseth, Assistant Professor, Université de Montréal, Core Academic Member, Mila

Research Labs

Mila professors exploring the subject as part of their research.

Mila Faculty
Associate Academic Member
Portrait of Narges Armanfard
Associate Professor, McGill University, Department of Electrical and Computer Engineering
Core Academic Member
Portrait of Glen Berseth
Assistant Professor, Université de Montréal, Department of Computer Science and Operations Research
Canada CIFAR AI Chair
Associate Academic Member
Portrait of Gregory Dudek is unavailable
Full Professor and Research Director of Mobile Robotics Lab, McGill University, School of Computer Science
Affiliate Member
Portrait of Samira Ebrahimi Kahou
Associate Professor, University of Calgary, Deparment of Electrical and Software Engineering
Core Academic Member
Portrait of Amir-massoud Farahmand
Associate Professor, Polytechnique Montréal
Associate Academic Member
Portrait of Toby Dylan Hocking
Associate Professor, Université Sherbrooke, Department of Computer Science
Associate Academic Member
Portrait of Xue (Steve) Liu is unavailable
Full Professor, McGill University, School of Computer Science
Associate Academic Member
Portrait of David Meger
Associate Professor, McGill University, School of Computer Science
Core Academic Member
Portrait of AJung Moon
Associate professor, McGill University, Department of Electrical and Computer Engineering
Associate Academic Member
Portrait of Eilif B. Muller
Assistant Professor, Université de Montréal, Department of Neurosciences
Canada CIFAR AI Chair
Core Academic Member
Portrait of Derek Nowrouzezahrai
Associate Professor, McGill University, Department of Electrical and Computer Engineering
Canada CIFAR AI Chair
Associate Academic Member
Portrait of Borke Obada-Obieh is unavailable
Assistant Professor, McGill University, School of Computer Science
Core Academic Member
Portrait of Chris Pal
Full Professor, Polytechnique Montréal, Department of Computer Engineering and Software Engineering
Canada CIFAR AI Chair
Core Academic Member
Portrait of Liam Paull
Assistant Professor, Université de Montréal, Department of Computer Science and Operations Research
Canada CIFAR AI Chair
Associate Academic Member
Portrait of Jana Pavlasek
Polytechnique Montréal, Department of Computer and Software Engineering
Affiliate Member
Portrait of Louis Petit
Assistant Professor, Université de Sherbrooke, Department of Electrical and Computer Engineering
Core Academic Member
Portrait of Doina Precup
Associate Professor, McGill University, School of Computer Science
Canada CIFAR AI Chair
Associate Academic Member
Portrait of Isabeau Prémont-Schwarz
Assistant Professor, Université Laval, Computer science and software engineering
Affiliate Member
Portrait of Jing Ren is unavailable
Assistant professor, Université de Montréal, DIRO
Associate Academic Member
Portrait of Audrey Sedal
Assistant Professor, McGill University, Department of Mechanical Engineering
Affiliate Member
Portrait of Inna Sharf
Full Professor, McGill University, Department of Mechanical Engineering
Associate Academic Member
Portrait of Kaleem Siddiqi
Professor, McGill University, School of Computer Science
Associate Academic Member
Portrait of Yang Wang
Associate Professor, Concordia University, Computer science and software engineering
Affiliate Member
Portrait of Hanqing Zhao
Assistant Professor, Université Laval, Electrical and Computer Engineering

Featured Video

Prof. Glen Berseth studies how machine learning can be used to train more adaptable robots that could help humanity meet its most pressing challenges.

Publications

Correlated Read Noise Reduction in Infrared Arrays Using Deep Learning
Étienne Artigaud
Laurence Perreault Levasseur
René Doyon
We present a new procedure rooted in deep learning to construct science images from data cubes collected by astronomical instruments using H… (see more)xRG detectors in low-flux regimes. It improves on the drawbacks of the conventional algorithms to construct 2D images from multiple readouts by using the readout scheme of the detectors to reduce the impact of correlated readout noise. We train a convolutional recurrent neural network on simulated astrophysical scenes added to laboratory darks to estimate the flux on each pixel of science images. This method achieves a reduction of the noise on constructed science images when compared to standard flux-measurement schemes (correlated double sampling, up-the-ramp sampling), which results in a reduction of the error on the spectrum extracted from these science images. Over simulated data cubes created in a low signal-to-noise ratio regime where this method could have the largest impact, we find that the error on our constructed science images falls faster than a
Navigating Potholes with Geometry-Aware Sharpness Minimization
Sharpness-aware minimization (SAM) encourages flat minima by perturbing parameters along directions of high loss curvature, but treats all p… (see more)arameter directions uniformly, ignoring the underlying loss geometry. We introduce LLQR+SAM, which combines SAM with a learned preconditioner obtained from the recently proposed LLQR framework, a second-order method that recasts steepest descent as a layerwise linear-quadratic regulator problem. The preconditioner is updated sparsely and maintained as a slow exponential moving average, so it captures a smoothed, low-resolution picture of the loss landscape geometry. The SAM perturbation then operates on top of this learned geometry, probing curvature at a faster timescale. We show that this two-timescale structure is not merely a computational convenience: theoretically, the preconditioner amplifies the SAM escape signal in directions that are flat under the average geometry but locally sharp (potholes). Wide, flat basins, by contrast, remain stable. Empirically, LLQR+SAM gives consistent gains over both SAM and LLQR alone across standard vision and sequence modeling benchmarks, supporting the view that slow learned geometry and fast sharpness correction are genuinely complementary.

Related Topics