Publications

Freeze, Don't Clone: Fine-Tuning with Expressive Policies Without Retaining Offline Data
Sami Nur Islam
Johan Obando-Ceron
Offline-to-online reinforcement learning uses offline pretraining to accelerate subsequent online learning. While many methods assume the of… (voir plus)fline dataset remains available throughout fine-tuning, there are many settings where this is not the case. Prior work attributes failures in this setting primarily to critic recalibration under distribution shift. We identify a complementary actor-side failure: once the offline data disappears, continued behavioral cloning (BC) no longer preserves the offline behavior distribution and instead refits the behavioral model to the agent's limited, and nonstationary, online experience. We demonstrate that stopping online behavioral fitting while preserving the pretrained expressive prior substantially improves no-retention fine-tuning across implicit flow Q-learning (IFQL), flow Q-learning (FQL), and action-chunked RL (QC) on seven OGBench domains. We trace this failure to a behavioral feedback loop: continuing online behavioral cloning rapidly erases the pretrained prior, inducing representational collapse and action diversity loss that starves the critic of useful exploration. Freezing the prior breaks this cycle, preserving offline coverage while allowing value learning to drive continuous improvement.
Audio-R1: Audio Reasoning through Reinforcement Learning on Self-Supervised Tasks
Reasoning is increasingly important for large audio-language models (LALMs), yet remains relatively underexplored compared with text-only an… (voir plus)d vision-language models. We introduce Audio-R1, a reinforcement learning (RL) post-training framework that revisits audio self-supervised learning as a source of verifiable rewards for improving audio understanding and reasoning task performance. Audio-R1 uses answers automatically determined during the construction of self-supervised tasks to verify model responses and compute rewards. Experiments across multiple LALMs and diverse audio benchmarks show consistent performance gains, while ablations demonstrate the benefits of each self-supervised task and additional gains from joint training. While both supervised fine-tuning and Audio-R1 improve self-supervised task accuracy, Audio-R1 further improves performance on downstream tasks distinct from the training tasks. These results highlight the potential of RL post-training with self-supervised audio tasks to improve downstream performance and reasoning-trace quality.
Closed‐Loop Targeted Memory Reactivation During Sleep Spindles Enhances Motor Memory Consolidation
Prakriti Gupta
Vaishali Mutreja
Ovidiu Lungu
Latifa Lazzouni
Ella Gabitov
Habib Benali
Hugo R. Jourde
Emily B. J. Coffey
Jean‐Marc Lina
Geneviève Albouy
Bradley H. King
Arnaud Boutin
Julie Carrier
Julien Doyon
Sleep spindles are thought to facilitate motor memory consolidation by reactivating recently acquired memory traces during non-rapid eye mov… (voir plus)ement (NREM) sleep, yet most human evidence is correlational. Using a closed-loop targeted memory reactivation (CL-TMR) protocol, this study tested whether auditory cueing time-locked to spindles enhances the consolidation of a newly acquired motor sequence. Twenty-six young adults practiced two eight-element, explicitly known, sequences of finger movements, each paired with a distinct auditory cue. During subsequent NREM Stage 2 and 3 nighttime sleep, a portable deep-learning-based EEG device (Portiloop) detected spindles online and selectively replayed the cue associated with one sequence (TMR condition), while the other served as a within-subject non-reactivated control (non-TMR) condition. Behavioural performance was assessed before and after sleep. Greater overnight improvements in performance were observed for the reactivated as compared to the non-reactivated condition, hence demonstrating enhanced consolidation following spindle-locked CL-TMR. The magnitude of this TMR benefit was positively correlated with the percentage of stimulations delivered during true spindles. Furthermore, a higher percentage of stimulations occurring during spindle trains (i.e., two or more spindles separated by 6 s) was associated with larger TMR-induced gains, whereas a higher percentage during isolated spindles (> 6 s apart) was associated with reduced gains. These findings provide further evidence that spindle-timed reactivation supports motor memory consolidation in humans and suggest that spindle trains may represent a favourable physiological window for closed-loop interventions.
Closing the Modality Gap via Simplex-Constrained Representations
While multimodal contrastive models like CLIP and CLAP successfully learn shared representations, their embeddings typically exhibit a persi… (voir plus)stent \emph{modality gap}. We propose addressing this separation through a purely geometric approach: changing the embedding domain from the unit sphere,
DIGOM: Disentangled Interaction Graphs for Outcome prediction in Multimodal healthcare
Multimodal healthcare data contain relational information that extends beyond individual patient observations. Effective clinical outcome pr… (voir plus)ediction therefore requires not only integrating information from multiple modalities, but also modeling the relationships between them. Standard fusion approaches do not explicitly capture these interactions, while existing graph-based approaches typically model only a subset of them. We propose DIGOM (Disentangled Interaction Graphs for Outcome prediction in Multimodal healthcare), a graph-based framework that jointly models three complementary interaction types: Intra-patient cross-modal, Cross-patient same-modal, and Cross-patient cross-modal interactions. DIGOM learns a sparse multiplex graph with disentangled views, each encoding interactions between pairs of modalities across patients. Discrete adjacency matrices with patient-adaptive connectivity are sampled via a differentiable mechanism, enabling end-to-end learning and allowing the model to select informative relationships without imposing a fixed number of neighbors. A regularization objective maintains overall graph sparsity, while message passing integrates this relational context into patient representations for outcome prediction. We evaluate DIGOM on two multimodal healthcare datasets, MIMIC-IV and NIH-TB, across five clinical prediction tasks. DIGOM achieves the highest AUROC on all five tasks while using fewer connections than existing multi-interaction approaches. Ablation studies reveal that each interaction type's contribution varies across tasks, highlighting the need to adapt relational structures per task. DIGOM also demonstrates greater robustness to missing modalities, suggesting learned relationships provide complementary information when modalities are unavailable. Post-hoc analyses further reveal task- and outcome-specific connectivity patterns in the learned graphs.
Every Edge Needs a Quote: Building a Quote-Grounded Knowledge Graph of Healthcare Software Adoption with Language-Model Agents
Juan Felipe Duran
Arthur Tobler
Hospitals regularly buy software that clinicians and patients depend on. Because these purchases are costly, hard to reverse and can affect … (voir plus)patient care, buyers look for evidence of which peer institutions already use a product and which systems it connects to. This evidence is public but scattered across vendor websites, press releases and third-party pages. Language-model agents can now collect it at low cost, but the correctness of the extracted claims is difficult to establish, and a wrongly listed customer can mislead buyers and misrepresent an organisation. We present a pipeline that builds a knowledge graph of customer and integration relations from the open web under one rule: every edge must be supported by a verbatim quote from its source page. Pages come from a deterministic crawl of each vendor's site and a budgeted ReAct agent that searches the rest of the web. A permissive extractor proposes relations with quotes, and a layered grounding stage decides which to keep, using a referent step that reads the whole page when a quote names only one side, deterministic checks, and a panel of five model calls voting on nine criteria. On 61 vendors, the pipeline reaches 0.94 precision and 0.71 recall against a page-level key covering the 6,428 candidate relations proposed by its extractor, whose verdicts were drafted by one model and re-checked with a second. Against the golden reference, the marketplace's own curated list, it scores lower, but 89% of the relations it keeps that the reference lacks are stated on a page. A confidence score built from the panel's votes keeps more correct relations than the fixed rule at the same precision, and reaches 0.90 precision at 0.90 recall. We also discuss how narrow, bounded model calls, string-matched evidence and voting address long-context degradation and run-to-run variance.
Recommender regret for improving regularised RL estimation with bandits and MCTS
Regularised reinforcement learning (RL) algorithms rely on estimators, like Monte Carlo tree search or one-step look-ahead, to provide polic… (voir plus)y, value, or Q-value estimates in training and inference. However, existing bandit-based estimation frameworks evaluate sampling efficiency, not the quality of estimates. We address this by formalising estimate generation as a recommendation process, treating bandit-based sampling as an improvement operator. To evaluate the quality of a recommendation (estimate), we define regulariser-aware recommender regrets, analysing their lower bounds, and proving that simple algorithms like uniform sampling can achieve these bounds. Finally, we unify bandit- and search-based target improvement under recommender MCTS. Across synthetic trees, arcade games, and scientific discovery tasks, recommenders often yield faster target convergence and superior downstream performance. Our experiments characterise for the first time the potential of regularised MCTS in scientific applications.
Regularised RL improves sim-to-real transfer
Regularised reinforcement learning (RL) has provable robustness guarantees that should ease the transfer of policies from simulation to robo… (voir plus)ts, known as sim-to-real. Yet, weakly regularised algorithms like Proximal Policy Optimisation (PPO) remain the de facto choice in sim-to-real pipelines, while the practical benefits of regularisation for transfer remain poorly characterised. In this work, we study the sim-to-real performance of commonly used RL algorithms, with the aim of characterising the effects of regularisation on robotic performance. To that end, we derive Soft MellowMax Actor-Critic (SMM-AC), a novel regularised RL algorithm based on the soft mellowmax value function, which carries strong robustness guarantees. On MuJoCo continuous control benchmarks, SMM-AC is competitive with baselines like Soft Actor-Critic (SAC). On real robots, SMM-AC achieves higher return, while exhibiting greater robustness to perturbations in observations, the environment, and dynamics than baselines. Together, these results demonstrate that regularised RL improves real-world policy deployment through better robustness to the sim-to-real gap, and changing real-world conditions.
Extended generalization and improved safety with the System‐2 features of the Scientist AI
Abstract Neural networks usually answer queries with a single forward pass—an analog of Kahneman's System 1. In unfamiliar or high‐stake… (voir plus)s regimes, this can be brittle and overconfident. The Scientist AI framework augments a fast Predictor trained to approximate the Bayesian posterior with System 2 inference: structured probabilistic reasoning over intermediate natural‐language statements and causal hypotheses. The units that are generated and related via probabilistic dependencies are not the words or tokens but these Boolean statements about arbitrary properties of the world. Because inference‐time compute is spent only on the query at hand, marginalizing over latent statements can reduce computational uncertainty, produce better calibrated conditional probabilities and confidence intervals, and generate explanations as explicit argument graphs. With data transformed into these statements processed to make a syntactic difference between communication acts and factual hypotheses and the Predictor trained to be consequence‐invariant —insensitive to downstream effects of its outputs (Fornasiere et al., 2026)—this epistemic caution supports a safety case: the system is not imitating humans, has less incentive to acquire implicit goals, and is better suited as a guardrail for untrusted agents.
GeoSinkhorn Flow: Geometry-Aware Flow Matching for Perturbation-Conditioned Dynamics in Single cell Data Phenoscapes
Ke Xu
Alistair Wilkinson
Ni Shuang
Gerstein Mark
Christopher J. Tape
Single-cell technologies increasingly enable large-scale profiling of complex biological systems across many experimental perturbations, but… (voir plus) understanding how a system reorganizes across an entire space of conditions is difficult to interpret from single-cell embeddings alone. Phenoscapes address this by embedding entire cell-state distributions, rather than individual cells, as points in a data-driven landscape. Thus, a single dot represents one experimental condition rather than one cell, yielding a map of the perturbation or patient landscape that reveals how phenotype varies continuously across conditions. Phenoscapes remain a comparatively new and underexplored representation and existing static phenoscapes position experimental conditions relative to one another without specifying trajectories that describe how a cellular population moves from one condition to another. Flow- and trajectory-based models for single-cell data are numerous, but existing approaches operate at the level of individual cells, are not built to respect the geometry of a phenoscape, and typically cannot generalize their predicted flows to conditions not seen during training. Here we introduce GeoSinkhorn Flow, a perturbation-conditioned, geometry-aware flow-matching framework specialized designed to convert a static phenoscape into a dynamic phenoscape, in which each condition-level point is additionally connected by geodesic, condition-aware transport trajectories that are consistent with the phenoscape's underlying geometry. GeoSinkhorn Flow can also model transitions directly between time points without constructing a phenoscape. GeoSinkhorn Flow uses heat-diffusion geometry to define manifold-aware transport costs and trains a geometric autoencoder whose latent Euclidean distances stay aligned with the static phenoscape geometry while extending it to unseen cells. By explicitly conditioning the learned transport dynamics on perturbation-specific gene or drug embeddings, GeoSinkhorn Flow generalizes its trajectories to new measurement conditions, produces intermediate progression states along a trajectory, and extends predictions to unseen cells and held-out perturbations. An unbalanced optimal transport variant further learns growth or transition weights, so that changes in relative cell-state abundance or transition propensity are modeled together with state movement. Across synthetic manifolds, patient-derived organoid drug responses, Perturb-seq knockdowns, and time-resolved CyTOF signaling, GeoSinkhorn Flow recovered intrinsic manifold geometry, inferred biologically coherent response trajectories, and generalized to held-out drug combinations, perturbation genes, and time points. Together, these results establish GeoSinkhorn Flow as a perturbation-conditioned, phenoscape-consistent dynamical framework, distinct from standard flow-matching approaches, for converting static phenoscapes into dynamic ones that generalize across experimental conditions.
A Symmetry-Enabled Direct Quantum Protocol for Many-Body Green's Functions
Changhao Yi
We present a symmetry-enabled direct quantum protocol for computing many-body Green's functions, a central tool for studying strongly correl… (voir plus)ated quantum systems. Our protocol relies only on native time evolution and straightforward measurements available on current hardware platforms. By exploiting parity symmetry—satisfied by a broad class of Hamiltonians in condensed matter physics and quantum chemistry, including the Fermi–Hubbard and Heisenberg models—we introduce a tailored quench spectroscopy scheme that recovers both the real and imaginary parts of two-point time correlators, from which Green's functions can be reconstructed via efficient classical signal processing. We further develop a tailored symmetric quantum Gibbs sampler that prepares parity-resolved (symmetric and antisymmetric) thermal states, enabling finite-temperature extensions within the same framework. Finally, we show that the same symmetry-based measurement primitive extends naturally to out-of-time-ordered correlators (OTOCs). Our results provide a practical route to estimating symmetry-resolved dynamical correlators on near-term and early fault-tolerant quantum hardware.
Cyber-resilient automatic generation control against false data injection attacks under unknown load variations
Sawsan Sayed
Georgia Pierrou
Xiaozhe Wang
François Bouffard
False Data Injection Attacks (FDIAs) pose a serious cyber-physical threat to systems that rely on sensor-actuator communication. Automatic G… (voir plus)eneration Control (AGC) is particularly vulnerable because it adjusts generator outputs using frequency and tie-line power measurements. If these measurements are corrupted, the resulting control actions can cause power imbalances, equipment damage, and widespread blackouts. Conventional Bad Data Detection (BDD) methods may fail to identify stealthy FDIAs before their physical effects become significant. This paper proposes a robust state-estimation framework for resilient AGC operation under stealthy FDIAs and unknown load variations. The framework is built on an unbiased minimum-variance estimator, with unknown load variations and injected attacks modeled as separate unknown-input channels. This formulation decouples their effects and enables attack reconstruction without load measurements or forecasts. Detection is performed in real time using residual-based indicators, while mitigation is applied when attacks can be reconstructed reliably. An admissibility analysis determines the protection level for each measurement-attack subset. Admissible subsets satisfy the conditions required for convergent and unbiased attack reconstruction; attacks on these channels are detected, reconstructed, and mitigated in real time. Attacks outside the admissible set are not reconstructed. For the non-admissible measurement attacks and control-channel attacks evaluated, cross-model residual inconsistency triggers a safe fallback that disables AGC and returns generators to local frequency control. The proposed method is benchmarked against an Unknown Input Observer (UIO) and a Kalman-filter-based detector using two-area and three-area AGC systems, and is validated on a real-time co-simulation platform.