Giovanni Beltrame

Traditional reinforcement learning (RL) generates discrete control policies, assigning one action per cycle. These policies are usually impl… (see more)emented as in a fixed-frequency control loop. This rigidity presents challenges as optimal control frequency is task-dependent; suboptimal frequencies increase computational demands and reduce exploration efficiency. Variable Time Step Reinforcement Learning (VTS-RL) addresses these issues with adaptive control frequencies, executing actions only when necessary, thus reducing computational load and extending the action space to include action durations. In this paper we introduce the Multi-Objective Soft Elastic Actor-Critic (MOSEAC) method to perform VTS-RL, validating it through theoretical analysis and experimentation in simulation and on real robots. Results show faster convergence, better training results, and reduced energy consumption with respect to other variable- or fixed-frequency approaches.

2024-06-28

ArXiv (preprint)

MOSEAC: Streamlined Variable Time Step Reinforcement Learning

Yong Wang

2024-06-02

ArXiv (preprint)

Hierarchies define the scalability of robot swarms

Vivek Shankar Vardharajan

Karthik Soma

Sepand Dyanatkar

Pierre-Yves Lajoie

The emerging behaviors of swarms have fascinated scientists and gathered significant interest in the field of robotics. Traditionally, swarm… (see more)s are viewed as egalitarian, with robots sharing identical roles and capabilities. However, recent findings highlight the importance of hierarchy for deploying robot swarms more effectively in diverse scenarios. Despite nature's preference for hierarchies, the robotics field has clung to the egalitarian model, partly due to a lack of empirical evidence for the conditions favoring hierarchies. Our research demonstrates that while egalitarian swarms excel in environments proportionate to their collective sensing abilities, they struggle in larger or more complex settings. Hierarchical swarms, conversely, extend their sensing reach efficiently, proving successful in larger, more unstructured environments with fewer resources. We validated these concepts through simulations and physical robot experiments, using a complex radiation cleanup task. This study paves the way for developing adaptable, hierarchical swarm systems applicable in areas like planetary exploration and autonomous vehicles. Moreover, these insights could deepen our understanding of hierarchical structures in biological organisms.

2024-05-02

ArXiv (preprint)

Learning Control Barrier Functions and their application in Reinforcement Learning: A Survey

Maeva Guerrier

Hassan Fouad

Reinforcement learning is a powerful technique for developing new robot behaviors. However, typical lack of safety guarantees constitutes a … (see more)hurdle for its practical application on real robots. To address this issue, safe reinforcement learning aims to incorporate safety considerations, enabling faster transfer to real robots and facilitating lifelong learning. One promising approach within safe reinforcement learning is the use of control barrier functions. These functions provide a framework to ensure that the system remains in a safe state during the learning process. However, synthesizing control barrier functions is not straightforward and often requires ample domain knowledge. This challenge motivates the exploration of data-driven methods for automatically defining control barrier functions, which is highly appealing. We conduct a comprehensive review of the existing literature on safe reinforcement learning using control barrier functions. Additionally, we investigate various techniques for automatically learning the Control Barrier Functions, aiming to enhance the safety and efficacy of Reinforcement Learning in practical robot applications.

2024-04-21

ArXiv (preprint)

From the Lab to the Theater: An Unconventional Field Robotics Journey

Ali Imran

Vivek Shankar Vardharajan

Rafael Gomes Braga

Yann Bouteiller

Abdalwhab Abdalwhab

Matthis Di-Giacomo

Alexandra Mercader

David St-Onge

2024-04-10

ArXiv (preprint)

Reinforcement Learning with Elastic Time Steps

Yong Wang

2024-02-21

ArXiv (preprint)

Deployable Reinforcement Learning with Variable Control Rate

Yong Wang

2024-01-16

ArXiv (preprint)

An Addendum to NeBula: Toward Extending Team CoSTAR’s Solution to Larger Scale Environments

Ali Agha

Kyohei Otsu

Benjamin Morrell

David D. Fan

Sung-Kyun Kim

Muhammad Fadhil Ginting

Xianmei Lei

Jeffrey Edlund

Seyed Fakoorian

Amanda Bouman

Fernando Chavez

Taeyeon Kim

Gustavo J. Correa

Maira Saboia

Angel Santamaria-Navarro

Brett Lopez

Boseong Kim

Chanyoung Jung

Mamoru Sobue

Oriana Claudia Peltzer … (see 69 more)

Joshua Ott

Robert Trybula

Thomas Touma

Marcel Kaufmann

Tiago Stegun Vaquero

Torkom Pailevanian

Matteo Palieri

Yun Chang

Andrzej Reinke

Matthew Anderson

Frederik E.T. Schöller

Patrick Spieler

Lillian Clark

Avak Archanian

Kenny Chen

Hovhannes Melikyan

Anushri Dixit

Harrison Delecki

Daniel Pastor

Barry Ridge

Nicolas Marchal

Jose Uribe

Sharmita Dey

Kamak Ebadi

Kyle Coble

Alexander Nikitas Dimopoulos

Vivek Thangavelu

Vivek Shankar Vardharajan

Nicholas Palomo

Antoni Rosinol

Arghya Chatterjee

Christoforos Kanellakis

Bjorn Lindqvist

Micah Corah

Kyle Strickland

Ryan Stonebraker

Michael Milano

Christopher E. Denniston

Sami Sahnoune

Thomas Claudet

Seungwook Lee

Gautam Salhotra

Edward Terry

Rithvik Musuku

Robin Schmid

Tony Tran

Ara Kourchians

Justin Schachter

Hector Azpurua

Levi Resende

Arash Kalantari

Jeremy Nash

Josh Lee

Christopher Patterson

Jen Blank

Kartik Patath

Yuki Kubo

Ryan Alimo

Yasin Almalioglu

Aaron Curtis

Jacqueline Sly

Tesla Wells

Nhut T. Ho

Mykel Kochenderfer

George Nikolakopoulos

David Shim

Luca Carlone

Joel Burdick

2023-12-31

IEEE Transactions on Field Robotics (published)

Canadarm, Canadarm2, and Canadarm3: The Evolution of Canada's Iconic Robotic System and Its Impacts from Space Down to Earth

Yianni Hudon-Castillo

Jean-Christophe Lamanque

Marion Thénault

Katherine Zamudio-Turcotte

Sri Venkata Vathsala Musunuri

Auriane Thilloy

Olivier Leclair

Mohamed Amine Elforaici

Rafael Daigneault

Rachad Chazbek

2023-12-31

58th IAA History of Astronautics Symposium (published)

Feasibility of cognitive neuroscience data collection during a speleological expedition

Anita Paas

Hugo R. Jourde

Arnaud Brignol

Marie-Anick Savard

Zseyvfin Eyqvelle

Samuel Bassetto

Emily B.J. Coffey

In human cognitive neuroscience and neuropsychology studies, laboratory-based research tasks have been important to establish principles of … (see more)brain function and its relationship to behaviour; however, they differ greatly from real-life experiences. Several elements of real-life situations that impact human performance, such as stressors, are difficult or impossible to replicate in the laboratory. Expeditions offer unique possibilities for studying human cognition in complex environments that can transfer to other situations with similar features. For example, as caves share several of the physical and psychological challenges of safety-critical environments such as spaceflight, underground expeditions have been developed as an analogue for astronaut training purposes, suggesting that they might also be suitable for studying aspects of behaviour and cognition that cannot be fully examined under laboratory conditions. While a large range of topics and tools have been proposed for use in such environments, few have been evaluated in the field. We tested the feasibility of collecting human physiological, cognitive, and subjective experience data concerning brain state, sleep, cognitive workload, and fatigue, during a speleological expedition in a remote region. We document our approaches and challenges experienced, and provide recommendations and suggestions to aid future work. The data support the idea that cave expeditions are relevant naturalistic paradigms that offer unique possibilities for cognitive neuroscience to complement laboratory work and help improve human performance and safety in operational environments.

2023-12-13

bioRxiv (preprint)

PEACE: Prompt Engineering Automation for CLIPSeg Enhancement in Aerial Robotics

Haechan Mark Bong

Rongge Zhang

Ricardo de Azambuja

From industrial to space robotics, safe landing is an essential component for flight operations. With the growing interest in artificial int… (see more)elligence, we direct our attention to learning based safe landing approaches. This paper extends our previous work, DOVESEI, which focused on a reactive UAV system by harnessing the capabilities of open vocabulary image segmentation. Prompt-based safe landing zone segmentation using an open vocabulary based model is no more just an idea, but proven to be feasible by the work of DOVESEI. However, a heuristic selection of words for prompt is not a reliable solution since it cannot take the changing environment into consideration and detrimental consequences can occur if the observed environment is not well represented by the given prompt. Therefore, we introduce PEACE (Prompt Engineering Automation for CLIPSeg Enhancement), powering DOVESEI to automate the prompt generation and engineering to adapt to data distribution shifts. Our system is capable of performing safe landing operations with collision avoidance at altitudes as low as 20 meters using only monocular cameras and image segmentation. We take advantage of DOVESEI's dynamic focus to circumvent abrupt fluctuations in the terrain segmentation between frames in a video stream. PEACE shows promising improvements in prompt generation and engineering for aerial images compared to the standard prompt used for CLIP and CLIPSeg. Combining DOVESEI and PEACE, our system was able improve successful safe landing zone selections by 58.62% compared to using only DOVESEI. All the source code is open source and available online.

2023-09-28

ArXiv (preprint)

From Assistive Devices to Manufacturing Cobot Swarms

Monica Li

Bruno Belzile

Ali Imran

Lionel Birglen

David St-Onge

This paper provides an overview of the latest trends in robotics research and development, with a particular focus on applications in manufa… (see more)cturing and industrial settings. We highlight recent advances in robot design, including cutting-edge collaborative robot mechanics and advanced safety features, as well as exciting developments in perception and human-swarm interaction. By examining recent contributions from Kinova, a leading robotics company, we illustrate the differences between industry and academia in their approaches to developing innovative robotic systems and technologies that enhance productivity and safety in the workplace. Ultimately, this paper demonstrates the tremendous potential of robotics to revolutionize manufacturing and industrial operations, and underscores the crucial role of companies like Kinova in driving this transformation forward.

2023-08-27

IEEE International Symposium on Robot and Human Interactive Communication (published)