Giovanni Beltrame

Traditional reinforcement learning (RL) generates discrete control policies, assigning one action per cycle. These policies are usually impl… (see more)emented as in a fixed-frequency control loop. This rigidity presents challenges as optimal control frequency is task-dependent; suboptimal frequencies increase computational demands and reduce exploration efficiency. Variable Time Step Reinforcement Learning (VTS-RL) addresses these issues with adaptive control frequencies, executing actions only when necessary, thus reducing computational load and extending the action space to include action durations. In this paper we introduce the Multi-Objective Soft Elastic Actor-Critic (MOSEAC) method to perform VTS-RL, validating it through theoretical analysis and experimentation in simulation and on real robots. Results show faster convergence, better training results, and reduced energy consumption with respect to other variable- or fixed-frequency approaches.

2024-06-29

ArXiv (preprint)

MOSEAC: Streamlined Variable Time Step Reinforcement Learning

Dong Wang

2024-06-03

ArXiv (preprint)

Hierarchies define the scalability of robot swarms

Vivek Shankar Vardharajan

Karthik Soma

Sepand Dyanatkar

Pierre-Yves Lajoie

The emerging behaviors of swarms have fascinated scientists and gathered significant interest in the field of robotics. Traditionally, swarm… (see more)s are viewed as egalitarian, with robots sharing identical roles and capabilities. However, recent findings highlight the importance of hierarchy for deploying robot swarms more effectively in diverse scenarios. Despite nature's preference for hierarchies, the robotics field has clung to the egalitarian model, partly due to a lack of empirical evidence for the conditions favoring hierarchies. Our research demonstrates that while egalitarian swarms excel in environments proportionate to their collective sensing abilities, they struggle in larger or more complex settings. Hierarchical swarms, conversely, extend their sensing reach efficiently, proving successful in larger, more unstructured environments with fewer resources. We validated these concepts through simulations and physical robot experiments, using a complex radiation cleanup task. This study paves the way for developing adaptable, hierarchical swarm systems applicable in areas like planetary exploration and autonomous vehicles. Moreover, these insights could deepen our understanding of hierarchical structures in biological organisms.

2024-05-03

ArXiv (preprint)

Overcoming boundaries: Interdisciplinary challenges and opportunities in cognitive neuroscience

Arnaud Brignol

Anita Paas

Luis Sotelo-Castro

David St-Onge

Emily B.J. Coffey

2024-05-01

Neuropsychologia (published)

Learning Control Barrier Functions and their application in Reinforcement Learning: A Survey

Maeva Guerrier

Hassan Fouad

Reinforcement learning is a powerful technique for developing new robot behaviors. However, typical lack of safety guarantees constitutes a … (see more)hurdle for its practical application on real robots. To address this issue, safe reinforcement learning aims to incorporate safety considerations, enabling faster transfer to real robots and facilitating lifelong learning. One promising approach within safe reinforcement learning is the use of control barrier functions. These functions provide a framework to ensure that the system remains in a safe state during the learning process. However, synthesizing control barrier functions is not straightforward and often requires ample domain knowledge. This challenge motivates the exploration of data-driven methods for automatically defining control barrier functions, which is highly appealing. We conduct a comprehensive review of the existing literature on safe reinforcement learning using control barrier functions. Additionally, we investigate various techniques for automatically learning the Control Barrier Functions, aiming to enhance the safety and efficacy of Reinforcement Learning in practical robot applications.

2024-04-22

ArXiv (preprint)

From the Lab to the Theater: An Unconventional Field Robotics Journey

Ali Imran

Vivek Shankar Vardharajan

Rafael Gomes Braga

Yann Bouteiller

Abdalwhab Abdalwhab

Matthis Di-Giacomo

Alexandra Mercader

David St-Onge

2024-04-11

ArXiv (preprint)

Reinforcement Learning with Elastic Time Steps

Dong Wang

2024-02-22

ArXiv (preprint)

Deployable Reinforcement Learning with Variable Control Rate

Dong Wang

2024-01-17

ArXiv (preprint)

Feasibility of cognitive neuroscience data collection during a speleological expedition

Anita Paas

Hugo R. Jourde

Arnaud Brignol

Marie-Anick Savard

Zseyvfin Eyqvelle

Samuel Bassetto

Emily B.J. Coffey

2023-12-14

bioRxiv (preprint)

PEACE: Prompt Engineering Automation for CLIPSeg Enhancement in Aerial Robotics

Haechan Mark Bong

Rongge Zhang

Ricardo de Azambuja

From industrial to space robotics, safe landing is an essential component for flight operations. With the growing interest in artificial int… (see more)elligence, we direct our attention to learning based safe landing approaches. This paper extends our previous work, DOVESEI, which focused on a reactive UAV system by harnessing the capabilities of open vocabulary image segmentation. Prompt-based safe landing zone segmentation using an open vocabulary based model is no more just an idea, but proven to be feasible by the work of DOVESEI. However, a heuristic selection of words for prompt is not a reliable solution since it cannot take the changing environment into consideration and detrimental consequences can occur if the observed environment is not well represented by the given prompt. Therefore, we introduce PEACE (Prompt Engineering Automation for CLIPSeg Enhancement), powering DOVESEI to automate the prompt generation and engineering to adapt to data distribution shifts. Our system is capable of performing safe landing operations with collision avoidance at altitudes as low as 20 meters using only monocular cameras and image segmentation. We take advantage of DOVESEI's dynamic focus to circumvent abrupt fluctuations in the terrain segmentation between frames in a video stream. PEACE shows promising improvements in prompt generation and engineering for aerial images compared to the standard prompt used for CLIP and CLIPSeg. Combining DOVESEI and PEACE, our system was able improve successful safe landing zone selections by 58.62% compared to using only DOVESEI. All the source code is open source and available online.

2023-09-29

ArXiv (preprint)

From Assistive Devices to Manufacturing Cobot Swarms

Monica Li

Bruno Belzile

Ali Imran

Lionel Birglen

David St-Onge

This paper provides an overview of the latest trends in robotics research and development, with a particular focus on applications in manufa… (see more)cturing and industrial settings. We highlight recent advances in robot design, including cutting-edge collaborative robot mechanics and advanced safety features, as well as exciting developments in perception and human-swarm interaction. By examining recent contributions from Kinova, a leading robotics company, we illustrate the differences between industry and academia in their approaches to developing innovative robotic systems and technologies that enhance productivity and safety in the workplace. Ultimately, this paper demonstrates the tremendous potential of robotics to revolutionize manufacturing and industrial operations, and underscores the crucial role of companies like Kinova in driving this transformation forward.

2023-08-28

IEEE International Symposium on Robot and Human Interactive Communication (published)

Electromagnetic interference shielding in lightweight carbon xerogels

Biporjoy Sarkar

Floriane Miquet-Westphal

Sanyasi Bobbara

Ben George

David Dousset

Ke Wu

Fabio Cicoira

With the increasing use of high-frequency electronic and wireless devices, electromagnetic interference (EMI) has become a growing concern d… (see more)ue to its potential impact on both electronic devices and human health. In this study, we demonstrated the performance of lightweight, electrically conducting 3D resorcinol-formaldehyde carbon xerogels, of 2.4 mm thickness, as an EMI shieldin the frequency range of 10–15 GHz (X-Ku band). The brittle carbon xerogels revealed complex porous structures with irregularly shaped pores that were randomly distributed. Electrochemical characterization revealed that the material behaved as an electrical double-layer capacitor. The carbon xerogels displayed reflection-dominated (∼ 84%) shielding behavior, with a total EMI shielding effectiveness (SE) value of ∼ 61 dB. The absorption process also contributed (∼ 16%) to the total SE. This behavior is attributed to the carbon xerogels' complex porous network, which effectively suppresses EM waves.

2023-03-21

Materials Research Express (published)