Publications

Is Exploration or Optimization the Problem for Deep Reinforcement Learning?
Glen Berseth
2025-06-30
rl-conference.cc/RLC/2025/Workshop/Finding_the_Frame (publié)
openreview.net
openreview.net
Filter Equivariant Functions: A symmetric account of length-general extrapolation on lists
Owen Lewis
Neil Ghani
Andrew Joseph Dudzik
Christos Perivolaropoulos
Razvan Pascanu
Petar Veličković
2025-06-30
arXiv (publié)
doi.org
A Geometric Lens on RL Environment Complexity Based on Ricci Curvature
Ali Saheb Pasand
Pablo Samuel Castro
Pouya Bashivan
We introduce Ollivier-Ricci Curvature (ORC) as an information-geometric tool for analyzing the local structure of reinforcement learning (RL… (voir plus)) environments. We establish a novel connection between ORC and the Successor Representation (SR), enabling a geometric interpretation of environment dynamics decoupled from reward signals. Our analysis shows that states with positive and negative ORC values correspond to regions where random walks converge and diverge respectively, which are often critical for effective exploration. ORC is highly correlated with established environment complexity metrics, yet integrates naturally with standard RL frameworks based on SR and provides both global and local complexity measures. Leveraging this property, we propose an ORC-based intrinsic reward that guides agents toward divergent regions and away from convergent traps. Empirical results demonstrate that our curvature-driven reward substantially improves exploration performance across diverse environments, outperforming both random and count-based intrinsic baselines.
2025-06-30
rl-conference.cc/RLC/2025/Workshop/Finding_the_Frame (publié)
openreview.net
openreview.net
Harnessing agent-based frameworks in CellAgentChat to unravel cell–cell interactions from single-cell and spatial transcriptomics
Vishvak Raghavan
Yumin Zheng
Yue Li
Jun Ding
Understanding cell–cell interactions (CCIs) is essential yet challenging owing to the inherent intricacy and diversity of cellular dynamic… (voir plus)s. Existing approaches often analyze global patterns of CCIs using statistical frameworks, missing the nuances of individual cell behavior owing to their focus on aggregate data. This makes them insensitive in complex environments where the detailed dynamics of cell interactions matter. We introduce CellAgentChat, an agent-based model (ABM) designed to decipher CCIs from single-cell RNA sequencing and spatial transcriptomics data. This approach models biological systems as collections of autonomous agents governed by biologically inspired principles and rules. Validated across eight diverse single-cell data sets, CellAgentChat demonstrates its effectiveness in detecting intricate signaling events across different cell populations. Moreover, CellAgentChat offers the ability to generate animated visualizations of single-cell interactions and provides flexibility in modifying agent behavior rules, facilitating thorough exploration of both close and distant cellular communications. Furthermore, CellAgentChat leverages ABM features to enable intuitive in silico perturbations via agent rule modifications, facilitating the development of novel intervention strategies. This ABM method unlocks an in-depth understanding of cellular signaling interactions across various biological contexts, thereby enhancing in silico studies for cellular communication–based therapies.
2025-06-30
Genome Research (publié)
doi.org
Health data issues in Africa: time for digitization, standardization and harmonization
Abdoelnaser Degoot
Ismaël Koné
Shakuntala Baichoo
Mercy Ngungu
Nzisa Liku
Judit Kumuthini
Joyce Nakatumba‐Nabende
Foutse Khomh
Bubacarr Bah
This commentary discusses health data challenges in Africa, focusing on digitization, standardization, and harmonization as key solutions. I… (voir plus)t highlights how addressing these foundational issues can enable AI and data science to transform healthcare systems across the continent.
2025-06-30
Nature Communications (publié)
doi.org
HVAC-GRACE: Transferable Building Control via Heterogeneous Graph Neural Network Policies
Anaïs Berkes
Donna Vakalis
David Rolnick
Yoshua Bengio
Buildings consume 40% of global energy, with HVAC systems responsible for up to half of that demand. As energy use grows, optimizing HVAC ef… (voir plus)ficiency is critical to meeting climate goals. While reinforcement learning (RL) offers a promising alternative to rule-based control, real-world adoption is limited by poor sample efficiency and generalisation. We introduce HVAC-GRACE, a graph-based RL framework that models buildings as heterogeneous graphs and integrates spatial message passing directly into temporal GRU gates. This enables each zone to learn control actions informed by both its own history and its structural context. Our architecture supports zero-shot transfer by learning topology-agnostic functions—but initial experiments reveal that this benefit depends on sufficient conditioned zone connectivity to maintain gradient flow. These findings highlight both the promise and the architectural requirements of scalable, transferable RL for building control
2025-06-30
ICML.cc/2025/Workshop/CO-BUILD (poster)
openreview.net
openreview.net
LLMs and Stack Overflow discussions: Reliability, impact, and challenges
Leuson Da Silva
Jordan Samhi
Foutse Khomh
2025-06-30
Journal of Systems and Software (publié)
doi.org
arxiv.org
Longitudinal changes in brain asymmetry track lifestyle and disease
Karin Saltoun
B. T. Thomas Yeo
Lynn Paul
Jörn Diedrichsen
Danilo Bzdok
Human beings may have evolved the largest asymmetries of brain organization in the animal kingdom. Hemispheric left-vs-right specialization … (voir plus)is especially pronounced in species-unique capacities, including emotional processing such as facial judgments, language-based feats such as reading books, and creativity such as musical performances. We hence chart the largest longitudinal brain-imaging resource, and provide evidence that brain asymmetry changes continuously in a manner suggestive of neural plasticity throughout adulthood. In the UK Biobank population cohort, we demonstrate that whole-brain patterns of asymmetry changes show robust phenome-wide associations across 959 distinct variables spanning 11 categories. We also find that changes in brain asymmetry over years co-occur with changes among specific lifestyle markers. We uncover specific brain asymmetry changes which systematically co-occur with entering a new phase of life, namely retirement. Finally, we reveal relevance of evolving brain asymmetry within subjects to major disease categories across ~4500 total medical diagnoses. Our findings speak against the idea that asymmetrical neural systems are conserved throughout adulthood.
2025-06-30
Nature Communications (publié)
doi.org
Model approximation in MDPs with unbounded per-step cost
Berk Bozkurt
Aditya Mahajan
Ashutosh Nayyar
Yi Ouyang
We consider the problem of designing a control policy for an infinite-horizon discounted cost Markov decision process …
2025-06-30
IEEE Transactions on Automatic Control (publié)
doi.org
arxiv.org
A Modular Approach for Clinical SLMs Driven by Synthetic Data with Pre-Instruction Tuning, Model Merging, and Clinical-Tasks Alignment
Jean-Philippe Corbeil
Amin Dada
Jean-Michel Attendu
Asma Ben Abacha
Alessandro Sordoni
Lucas Caccia
Franccois Beaulieu
Thomas Lin
Jens Kleesiek
Paul Vozila
High computation costs and latency of large language models such as GPT-4 have limited their deployment in clinical settings. Small language… (voir plus) models (SLMs) offer a cost-effective alternative, but their limited capacity requires biomedical domain adaptation, which remains challenging. An additional bottleneck is the unavailability and high sensitivity of clinical data. To address these challenges, we propose a novel framework for adapting SLMs into high-performing clinical models. We introduce the MediPhi collection of 3.8B-parameter SLMs developed with our novel framework: pre-instruction tuning of experts on relevant medical and clinical corpora (PMC, Medical Guideline, MedWiki, etc.), model merging, and clinical-tasks alignment. To cover most clinical tasks, we extended the CLUE benchmark to CLUE+, doubling its size. Our expert models deliver relative improvements on this benchmark over the base model without any task-specific fine-tuning: 64.3% on medical entities, 49.5% on radiology reports, and 44% on ICD-10 coding (outperforming GPT-4-0125 by 14%). We unify the expert models into MediPhi via model merging, preserving gains across benchmarks. Furthermore, we built the MediFlow collection, a synthetic dataset of 2.5 million high-quality instructions on 14 medical NLP tasks, 98 fine-grained document types, and JSON format support. Alignment of MediPhi using supervised fine-tuning and direct preference optimization achieves further gains of 18.9% on average.
2025-06-30
Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (publié)
doi.org
arxiv.org
Modulation of leg trajectory by transcranial magnetic stimulation during walking
Héloïse Bourgeois
Rose Guay-Hottin
El-Mehdi Meftah
Marina Martinez
Marco Bonizzato
Dorothy Barthélemy
The primary motor cortex is involved in initiation and adaptive control of locomotion. However, the role of the motor cortex in controlling … (voir plus)gait trajectories remains unclear. In animals, cortical neuromodulation allows for precise control of step height. We hypothesized that a similar control framework applies to humans, whereby cortical stimulation would primarily increase foot elevation. Transcranial magnetic stimulation (TMS) was applied over the motor cortex to assess the involvement of the corticospinal tract over the limb trajectory during human walking. Ten healthy adults (aged 20–32 years) participated in treadmill walking at 1.5 km/h. TMS was applied over the left motor cortex at an intensity of 120% of the threshold to elicit a dorsiflexion of the right ankle during the swing phase of gait. Electromyographic (EMG) measurements and three-dimensional (3D) lower limb kinematics were collected. When delivered during the early swing phase, TMS led to a significant increase in the maximum height of the right toe by a mean of 34.9% ± 9.6% (21.4 mm ± 7.9 mm, p = 0.032) and knee height by 52.8% ± 14.1% (28.8 mm ± 7.7 mm, p = 0.0021) across participants. These findings indicate that TMS can influence limb trajectory during walking, highlighting its potential as a tool for studying cortical control of locomotion.
2025-06-30
Scientific Reports (publié)
doi.org
MSR37 Improve Analyst Accuracy in Systematic Literature Reviews Using Reliant Tabular and LLM-Based Relevance Scoring
Christoph R. Schlegel
Sam Work
Bellemare Marc-Emmanuel
2025-06-30
Value in Health (publié)
doi.org