Giovanni Beltrame

PhD - Polytechnique Montréal

Co-supervisor :

Collaborating researcher - Polytechnique Montréal Montreal

Co-supervisor :

Master's Research - Polytechnique Montréal

Co-supervisor :

PhD - Polytechnique Montréal

Co-supervisor :

Jana Pavlasek

Github

Simon Roy

Master's Research - Université de Montréal

Co-supervisor :

Jana Pavlasek

Website

Github

Soma Soma

PhD - Polytechnique Montréal

Co-supervisor :

Jana Pavlasek

Publications

3D Foundation Model-Based Loop Closing for Decentralized Collaborative SLAM

Pierre-Yves Lajoie

Benjamin Ramtoula

Daniele De Martini

Decentralized Collaborative Simultaneous Localization and Mapping (C-SLAM) techniques often struggle to identify map overlaps due to signifi… (see more)cant viewpoint variations among robots. Motivated by recent advancements in 3D foundation models, which can register images despite large viewpoint differences, we propose a robust loop closing approach that leverages these models to establish inter-robot measurements. In contrast to resource-intensive methods requiring full 3D reconstruction within a centralized map, our approach integrates foundation models into existing SLAM pipelines, yielding scalable and robust multi-robot mapping. Our contributions include: 1) integrating 3D foundation models to reliably estimate relative poses from monocular image pairs within decentralized C-SLAM; 2) introducing robust outlier mitigation techniques critical to the use of these relative poses and 3) developing specialized pose graph optimization formulations that efficiently resolve scale ambiguities. We evaluate our method against state-of-the-art approaches, demonstrating improvements in localization and mapping accuracy, alongside significant gains in computational and memory efficiency. These results highlight the potential of our approach for deployment in large-scale multi-robot scenarios.

2024-12-31

IEEE Robotics Autom. Lett. (published)

Multi-Robot Decentralized Collaborative SLAM in Planetary Analogue Environments: Dataset, Challenges, and Lessons Learned

Pierre-Yves Lajoie

Karthik Soma

Alice Lemieux-Bourque

Rongge Zhang

Vivek Shankar Varadharajan

Decentralized collaborative simultaneous localization and mapping (C-SLAM) is essential to enable multirobot missions in unknown environment… (see more)s without relying on preexisting localization and communication infrastructure. This technology is anticipated to play a key role in the exploration of the Moon, Mars, and other planets. In this article, we share insights and lessons learned from C-SLAM experiments involving three robots operating on a Mars analogue terrain and communicating over an ad hoc network. We examine the impact of limited and intermittent communication on C-SLAM performance, as well as the unique localization challenges posed by planetary-like environments. Additionally, we introduce a novel dataset collected during our experiments, which includes real-time peer-to-peer inter-robot throughput and latency measurements. This dataset aims to support future research on communication-constrained, decentralized multirobot operations.

2024-12-31

IEEE Transactions on Field Robotics (published)

A Multi-Robot Exploration Planner for Space Applications

Vivek Shankar Vardharajan

We propose a distributed multi-robot exploration planning method designed for complex, unconstrained environments featuring steep elevation … (see more)changes. The method employs a two-tiered approach: a local exploration planner that constructs a grid graph to maximize exploration gain and a global planner that maintains a sparse navigational graph to track visited locations and frontier information. The global graphs are periodically synchronized among robots within communication range to maintain an updated representation of the environment. Our approach integrates localization loop closure estimates to correct global graph drift. In simulation and field tests, the proposed method achieves 50% lower computational runtime compared to state-of-the-art methods while demonstrating superior exploration coverage. We evaluate its performance in two simulated subterranean environments and in field experiments at a Mars-analog terrain.

2024-12-31

IEEE Robotics and Automation Letters (published)

Neural Incremental Dynamic Inversion Control of a Multirotor Robotic Airship

Ely Carneiro de Paiva

José Raul Azinheira

Rafael de Angelis Cordeiro

José Reginaldo H. Carvalho

Apolo Marton

2024-12-31

International Journal of Intelligent Systems (published)

PEACE: Prompt Engineering Automation for CLIPSeg Enhancement for Safe-Landing Zone Segmentation

Rongge Zhang

Antoine Robillard

Safe landing is essential in robotics applications, from industrial settings to space exploration. As artificial intelligence advances, we h… (see more)ave developed PEACE (Prompt Engineering Automation for CLIPSeg Enhancement), a system that automatically generates and refines prompts for identifying landing zones in changing environments. Traditional approaches using fixed prompts for open-vocabulary models struggle with environmental changes and can lead to dangerous outcomes when conditions are not represented in the predefined prompts. PEACE addresses this limitation by dynamically adapting to shifting data distributions. Our key innovation is the dual segmentation of safe and unsafe landing zones, allowing the system to refine the results by removing unsafe areas from potential landing sites. Using only monocular cameras and image segmentation, PEACE can safely guide descent operations from 100 meters to altitudes as low as 20 meters. The testing shows that PEACE significantly outperforms the standard CLIP and CLIPSeg prompting methods, improving the successful identification of safe landing zones from 57% to 92%. We have also demonstrated enhanced performance when replacing CLIPSeg with FastSAM. The complete source code is available as an open-source software 1.

2024-12-31

IROS (published)

BlabberSeg: Real-Time Embedded Open-Vocabulary Aerial Segmentation

Ricardo de Azambuja

Real-time aerial image segmentation plays an important role in the environmental perception of Uncrewed Aerial Vehicles (UAVs). We introduce… (see more) BlabberSeg, an optimized Vision-Language Model built on CLIPSeg for on-board, real-time processing of aerial images by UAVs. BlabberSeg improves the efficiency of CLIPSeg by reusing prompt and model features, reducing computational overhead while achieving real-time open-vocabulary aerial segmentation. We validated BlabberSeg in a safe landing scenario using the Dynamic Open-Vocabulary Enhanced SafE-Landing with Intelligence (DOVESEI) framework, which uses visual servoing and open-vocabulary segmentation. BlabberSeg reduces computational costs significantly, with a speed increase of 927.41% (16.78 Hz) on a NVIDIA Jetson Orin AGX (64GB) compared with the original CLIPSeg (1.81Hz), achieving real-time aerial segmentation with negligible loss in accuracy (2.1% as the ratio of the correctly segmented area with respect to CLIPSeg). BlabberSeg's source code is open and available online.

2024-10-15

ArXiv (preprint)

Active Semantic Mapping and Pose Graph Spectral Analysis for Robot Exploration

Rongge Zhang

2024-10-13

2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (published)

Physical Simulation for Multi-agent Multi-machine Tending

Abdalwhab Abdalwhab

David St-Onge

2024-10-10

ArXiv (preprint)

Beyond the lab: Feasibility of cognitive neuroscience data collection during a speleological expedition

Anita Paas

Hugo R. Jourde

Arnaud Brignol

Marie-Anick Savard

Zseyvfin Eyqvelle

Samuel Bassetto

Emily B.J. Coffey

2024-10-04

Journal of Environmental Psychology (published)

Multi-Objective Risk Assessment Framework for Exploration Planning Using Terrain and Traversability Analysis

Riana Gagnon Souleiman

Vivek Shankar Vardharajan

2024-10-03

ArXiv (preprint)

Frequency-based View Selection in Gaussian Splatting Reconstruction

Monica Li

Pierre-Yves Lajoie

Three-dimensional reconstruction is a fundamental problem in robotics perception. We examine the problem of active view selection to perform… (see more) 3D Gaussian Splatting reconstructions with as few input images as possible. Although 3D Gaussian Splatting has made significant progress in image rendering and 3D reconstruction, the quality of the reconstruction is strongly impacted by the selection of 2D images and the estimation of camera poses through Structure-from-Motion (SfM) algorithms. Current methods to select views that rely on uncertainties from occlusions, depth ambiguities, or neural network predictions directly are insufficient to handle the issue and struggle to generalize to new scenes. By ranking the potential views in the frequency domain, we are able to effectively estimate the potential information gain of new viewpoints without ground truth data. By overcoming current constraints on model architecture and efficacy, our method achieves state-of-the-art results in view selection, demonstrating its potential for efficient image-based 3D reconstruction.

2024-09-23

ArXiv (preprint)

Swarming Out of the Lab: Comparing Relative Localization Methods for Collective Behavior

Rafael Gomes Braga

Vivek Shankar Vardharajan

David St-Onge

2024-09-14

Lecture Notes in Computer Science (published)