Publications

Torque-Based Deep Reinforcement Learning for Task-and-Robot Agnostic Learning on Bipedal Robots Using Sim-to-Real Transfer
Donghyeon Kim
Glen Berseth
Mathew Schwartz
Jaeheung Park
In this letter, we review the question of which action space is best suited for controlling a real biped robot in combination with Sim2Real … (see more)training. Position control has been popular as it has been shown to be more sample efficient and intuitive to combine with other planning algorithms. However, for position control, gain tuning is required to achieve the best possible policy performance. We show that, instead, using a torque-based action space enables task-and-robot agnostic learning with less parameter tuning and mitigates the sim-to-reality gap by taking advantage of torque control's inherent compliance. Also, we accelerate the torque-based-policy training process by pre-training the policy to remain upright by compensating for gravity. The letter showcases the first successful sim-to-real transfer of a torque-based deep reinforcement learning policy on a real human-sized biped robot.
2023-10-01
IEEE Robotics and Automation Letters (published)
doi.org
arxiv.org
Towards More General Loss and Setting in Unsupervised Domain Adaptation
Changjian Shui
Ruizhi Pu
Gezheng Xu
Jun Wen
Fan Zhou
Christian Gagné
Charles Ling
Boyu Wang
In this article, we present an analysis of unsupervised domain adaptation with a series of theoretical and algorithmic results. We derive a … (see more)novel Rényi-
2023-10-01
IEEE Transactions on Knowledge and Data Engineering (published)
doi.org
W56. UNRAVELING THE IMPACT OF GENOMIC VARIATIONS ON COGNITIVE ABILITY ACROSS THE HUMAN CORTEX: INSIGHTS FROM GENE EXPRESSION AND COPY NUMBER VARIANTS
Kuldeep Kumar
Sayeh Kazem
Guillaume Huguet
Thomas Renne
Bank Engchuan
Omar Shanta
Bhooma Thiruvahindrapuram
J. MacDonald
Marieke Klein
Stephen W Scherer
Laura Almasy
Jonathan Sebat
David C. Glahn
Guillaume Dumas
Sébastien Jacquemont
2023-10-01
European Neuropsychopharmacology (published)
doi.org
Zero-Shot Fault Detection for Manipulators Through Bayesian Inverse Reinforcement Learning
Hanqing Zhao
Xue (Steve) Liu
Gregory Dudek
We consider the detection of faults in robotic manipulators, with particular emphasis on faults that have not been observed or identified in… (see more) advance, which naturally includes those that occur very infrequently. Recent studies indicate that the reward function obtained through Inverse Reinforcement Learning (IRL) can help detect anomalies caused by faults in a control system (i.e. fault detection). Current IRL methods for fault detection, however, either use a linear reward representation or require extensive sampling from the environment to estimate the policy, rendering them inappropriate for safety-critical situations where sampling of failure observations via fault injection can be expensive and dangerous. To address this issue, this paper proposes a zero-shot and exogenous fault detector based on an approximate variational reward imitation learning (AVRIL) structure. The fault detector recovers a reward signal as a function of externally observable information to describe the normal operation, which can then be used to detect anomalies caused by faults. Our method incorporates expert knowledge through a customizable reward prior distribution, allowing the fault detector to learn the reward solely from normal operation samples, without the need for a simulator or costly interactions with the environment. We evaluate our approach for exogenous partial fault detection in multi-stage robotic manipulator tasks, comparing it with several baseline methods. The results demonstrate that our method more effectively identifies unseen faults even when they occur within just three controller time steps.
2023-10-01
2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (published)
doi.org
Effectiveness of regional diffusion MRI measures in distinguishing multiple sclerosis abnormalities within the cervical spinal cord
Haykel Snoussi
Julien Cohen-Adad
Olivier Commowick
Benoit Combes
Elise Bannier
Slimane Tounekti
Anne Kerbrat
Christian Barillot
Emmanuel Caruyer
Multiple sclerosis (MS) is an inflammatory disorder of the central nervous system. Although conventional magnetic resonance imaging (MRI) is… (see more) widely used for MS diagnosis and clinical follow‐up, quantitative MRI has the potential to provide valuable intrinsic values of tissue properties that can enhance accuracy. In this study, we investigate the efficacy of diffusion MRI in distinguishing MS lesions within the cervical spinal cord, using a combination of metrics extracted from diffusion tensor imaging and Ball‐and‐Stick models.
2023-09-29
Brain and Behavior (published)
doi.org
arxiv.org
PEACE: Prompt Engineering Automation for CLIPSeg Enhancement in Aerial Robotics
Haechan Mark Bong
Rongge Zhang
Ricardo de Azambuja
Giovanni Beltrame
From industrial to space robotics, safe landing is an essential component for flight operations. With the growing interest in artificial int… (see more)elligence, we direct our attention to learning based safe landing approaches. This paper extends our previous work, DOVESEI, which focused on a reactive UAV system by harnessing the capabilities of open vocabulary image segmentation. Prompt-based safe landing zone segmentation using an open vocabulary based model is no more just an idea, but proven to be feasible by the work of DOVESEI. However, a heuristic selection of words for prompt is not a reliable solution since it cannot take the changing environment into consideration and detrimental consequences can occur if the observed environment is not well represented by the given prompt. Therefore, we introduce PEACE (Prompt Engineering Automation for CLIPSeg Enhancement), powering DOVESEI to automate the prompt generation and engineering to adapt to data distribution shifts. Our system is capable of performing safe landing operations with collision avoidance at altitudes as low as 20 meters using only monocular cameras and image segmentation. We take advantage of DOVESEI's dynamic focus to circumvent abrupt fluctuations in the terrain segmentation between frames in a video stream. PEACE shows promising improvements in prompt generation and engineering for aerial images compared to the standard prompt used for CLIP and CLIPSeg. Combining DOVESEI and PEACE, our system was able improve successful safe landing zone selections by 58.62% compared to using only DOVESEI. All the source code is open source and available online.
2023-09-29
ArXiv (preprint)
doi.org
arxiv.org
Tree Cross Attention
Leo Feng
Frederick Tung
Hossein Hajimirsadeghi
Yoshua Bengio
Mohamed Osama Ahmed
2023-09-29
ArXiv (preprint)
doi.org
arxiv.org
An Empirical Study of Retrieval-Enhanced Graph Neural Networks
Dingmin Wang
Shengchao Liu
Hanchen Wang
Bernardo Cuenca Grau
Linfeng Song
Jian Tang
Le Song
Qi Liu
Graph Neural Networks (GNNs) are effective tools for graph representation learning. Most GNNs rely on a recursive neighborhood aggregation s… (see more)cheme, named message passing, thereby their theoretical expressive power is limited to the first-order Weisfeiler-Lehman test (1-WL). An effective approach to this challenge is to explicitly retrieve some annotated examples used to enhance GNN models. While retrieval-enhanced models have been proved to be effective in many language and vision domains, it remains an open question how effective retrieval-enhanced GNNs are when applied to graph datasets. Motivated by this, we want to explore how the retrieval idea can help augment the useful information learned in the graph neural networks, and we design a retrieval-enhanced scheme called GRAPHRETRIEVAL, which is agnostic to the choice of graph neural network models. In GRAPHRETRIEVAL, for each input graph, similar graphs together with their ground-true labels are retrieved from an existing database. Thus they can act as a potential enhancement to complete various graph property predictive tasks. We conduct comprehensive experiments over 13 datasets, and we observe that GRAPHRETRIEVAL is able to reach substantial improvements over existing GNNs. Moreover, our empirical study also illustrates that retrieval enhancement is a promising remedy for alleviating the long-tailed label distribution problem.
2023-09-28
Frontiers in Artificial Intelligence and Applications (published)
doi.org
arxiv.org
Influence of preprocessing, distortion correction and cardiac triggering on the quality of diffusion MR images of spinal cord
Kurt G Schilling
Anna J. E. Combes
Karthik Ramadass
Francois Rheault
Grace Sweeney
Logan Prock
Subramaniam Sriram
Julien Cohen-Adad
John C Gore
Bennett A Landman
Seth A. Smith
Kristin P. O’Grady
Diffusion MRI of the spinal cord (SC) is susceptible to geometric distortion caused by field inhomogeneities, and prone to misalignment acro… (see more)ss time series and signal dropout caused by biological motion. Several modifications of image acquisition and image processing techniques have been introduced to overcome these artifacts, but their specific benefits are largely unproven and warrant further investigations. We aim to evaluate two specific aspects of image acquisition and processing that address image quality in diffusion studies of the spinal cord: susceptibility corrections to reduce geometric distortions, and cardiac triggering to minimize motion artifacts. First, we evaluate 4 distortion preprocessing strategies on 7 datasets of the cervical and lumbar SC and find that while distortion correction techniques increase geometric similarity to structural images, they are largely driven by the high-contrast cerebrospinal fluid, and do not consistently improve the geometry within the cord nor improve white-to-gray matter contrast. We recommend at a minimum to perform bulk-motion correction in preprocessing and posit that improvements/adaptations are needed for spinal cord distortion preprocessing algorithms, which are currently optimized and designed for brain imaging. Second, we design experiments to evaluate the impact of removing cardiac triggering. We show that when triggering is foregone, images are qualitatively similar to triggered sequences, do not have increased prevalence of artifacts, and result in similar diffusion tensor indices with similar reproducibility to triggered acquisitions. When triggering is removed, much shorter acquisitions are possible, which are also qualitatively and quantitatively similar to triggered sequences. We suggest that removing cardiac triggering for cervical SC diffusion can be a reasonable option to save time with minimal sacrifice to image quality.
2023-09-27
bioRxiv (preprint)
doi.org
RECOVER identifies synergistic drug combinations in vitro through sequential model optimization
Paul Bertin
Jarrid Rector-Brooks
Deepak Sharma
Thomas Gaudelet
Andrew Anighoro
Torsten Gross
Francisco Martínez-Peña
Eileen L. Tang
M.S. Suraj
Cristian Regep
Jeremy B.R. Hayter
Maksym Korablyov
Nicholas Valiante
Almer van der Sloot
Mike Tyers
Charles E.S. Roberts
Michael M. Bronstein
Luke L. Lairson
Jake P. Taylor-King
Yoshua Bengio
2023-09-27
Cell Reports Methods (published)
doi.org
Time Delay Cosmography with a Neural Ratio Estimator
Eve Campeau-Poirier
Laurence Perreault-Levasseur
Adam Coogan
Yashar Hezaveh
We explore the use of a Neural Ratio Estimator (NRE) to determine the Hubble constant (…
2023-09-27
ArXiv (preprint)
arxiv.org
arxiv.org
ClimateSet: A Large-Scale Climate Model Dataset for Machine Learning
Julia Kaltenborn
Charlotte Emilie Elektra Lange
Venkatesh Ramesh
Philippe Brouillard
Yaniv Gurwicz
Chandni Nagda
Jakob Runge
Peer Nowack
David Rolnick
Climate models have been key for assessing the impact of climate change and simulating future climate scenarios. The machine learning (ML) c… (see more)ommunity has taken an increased interest in supporting climate scientists’ efforts on various tasks such as climate model emulation, downscaling, and prediction tasks. Many of those tasks have been addressed on datasets created with single climate models. However, both the climate science and ML communities have suggested that to address those tasks at scale, we need large, consistent, and ML-ready climate model datasets. Here, we introduce ClimateSet, a dataset containing the inputs and outputs of 36 climate models from the Input4MIPs and CMIP6 archives. In addition, we provide a modular dataset pipeline for retrieving and preprocessing additional climate models and scenarios. We showcase the potential of our dataset by using it as a benchmark for ML-based climate model emulation. We gain new insights about the performance and generalization capabilities of the different ML models by analyzing their performance across different climate models. Furthermore, the dataset can be used to train an ML emulator on several climate models instead of just one. Such a “super-emulator” can quickly project new climate change scenarios, complementing existing scenarios already provided to policymakers. We believe ClimateSet will create the basis needed for the ML community to tackle climate-related tasks at scale.
openreview.net