Publications

Unifying Local Communications and Local Updates for LLM Pretraining
Pietro Cagnasso
Eugene Belilovsky
Edouard Oyallon
Communication-efficient pre-training of LLMs is increasingly important as training draws on compute distributed across clusters, data center… (see more)s, and lower-bandwidth links. Many practical methods reduce communication frequency but still rely on synchronous All-Reduce operations that maintain identical model states and tie progress to global collectives. This can become a bottleneck when bandwidth or worker speed is heterogeneous. We introduce GASLoC, a novel decentralized pre-training algorithm that generalizes the notion of communication acceleration to the recently popular"outer optimizer"to allow a practical gossip-based training framework that is compatible with adaptive optimizers, allows for local optimizer steps, and can utilize sparse randomized peer communication. Empirically, on a number of standard LLM training tasks, we demonstrate that GASLoC outperforms state-of-the-art decentralized algorithms in single step per communication setting for a number of topologies and, unlike existing decentralized methods in the LLM setting, it allows to obtain performance competitive with DiLoCo when utilizing multiple local steps. In the heterogeneous bandwidth setting we demonstrate the advantage of GASLoC showing that it can significantly outperform DiLoCo.
2026-06-08
arXiv (preprint)
doi.org
arxiv.org
Charting Cervical Spinal Cord Morphometry Across the Lifespan
Kurt Schilling
Michael E Kim
Matthew Amandola
Chenyu Gao
Karthik Ramadass
Praitayini Kanakaraj
Sam Bogdanov
G Rudravaram
Nancy R. Newlin
Derek B. Archer
Timothy J Hohman
Angela L Jefferson
Victoria L Morgan
Alexandra Roche
Dario J Englot
Murat Bilgel
Lori L Beason-Held
Luigi Ferrucci
Laurie Cutting
Laura A Barquero … (see 21 more)
Micah D’Archangel
Tin Q Nguyen
Kathryn L Humphreys
Yanbin Niu
Sophia Vinci-Booher
Carissa J. Cascio
Zhiyuan Li
Daniel Moyer
Simon Vandekar
Panpan Zhang
Samuelle St-Onge
Sandrine Bédard
Jan Valosek
Benjamin De Leener
Julien Cohen-Adad
John C Gore
Seth Smith
B A Landman
John C. Gore
Seth Smith
Bennett A. Landman
Abstract Spinal cord morphometry provides essential biomarkers of neurological health, but clinical interpretations are confounded by inter-… (see more)subject variability and a lack of normative references across the full human lifespan. We address this gap by generating the first comprehensive lifespan charts for cervical spinal cord morphometry. We leveraged 30 population-based brain MRI datasets, aggregating 78,269 scans from 41,042 individuals (ages 0–100) whose imaging protocols included cervical cord coverage. To overcome contrast variability, we employed a state-of-the-art contrast-agnostic deep learning segmentation method, extracting cross-sectional area (CSA), anteroposterior (AP) and right–left (RL/transverse), and shape indices (compression ratio, eccentricity, and solidity) from C1 to C7. Normative trajectories were modeled using Generalized Additive Models for Location, Scale, and Shape (GAMLSS). The resulting charts reveal distinct non-linear lifespan changes: rapid growth through childhood and adolescence, peak maturation occurring in early-to-mid adulthood (e.g., mid-30s for CSA), followed by gradual decreases. Significant regional variations along the cervical cord and consistent sex differences (males > females for size metrics) were quantified. Spinal cord trajectories showed strong temporal coupling with brain white matter and brainstem volumes, suggesting integrated CNS development and aging. These lifespan charts provide a robust normative framework, enabling age- and sex-specific centile scoring of individual spinal cord morphometry. This resource offers a critical tool for differentiating typical variation from pathological changes, enhancing the clinical utility of spinal cord MRI in studies of development and neurodegeneration.
2026-06-07
bioRxiv (accepted)
doi.org
Difference-Aware Retrieval Policies for Imitation Learning
Quinn Pfeifer
Ethan Pronovost
Paarth Shah
Khimya Khetarpal
Siddhartha Srinivasa
Abhishek Gupta
Parametric imitation learning via behavior cloning can suffer from poor generalization to out-of-distribution states due to compounding erro… (see more)rs during deployment. We show that reusing the training data during inference via a semi-parametric retrieval-based imitation learning approach can alleviate this challenge. We present Difference-Aware Retrieval Policies for Imitation Learning (DARP), a semi-parametric retrieval-based imitation learning approach that addresses this limitation by reparameterizing the imitation learning problem in terms of local neighborhood structure rather than direct state-to-action mappings. Instead of learning a global policy, DARP trains a model to predict actions based on
2026-06-07
arXiv (preprint)
doi.org
arxiv.org
OpenBibleTTS: Large-Scale Speech Resources and TTS Models for Low-Resource Languages
David Guzmán
Luel Hagos Beyene
Jesujoba Oluwadara Alabi
Yejin Jeon
Dietrich Klakow
David Ifeoluwa Adelani
Recent advances in neural text-to-speech (TTS) and multilingual speech generation have substantially improved synthetic speech quality, yet … (see more)these gains remain unevenly distributed across the world's languages. Existing models are still dominated by a small set of high-resource languages, while many studies of low-resource TTS are simulated on artificially downsampled high-resource corpora that do not reflect the orthographic variation and limited phonetic coverage encountered in genuinely underrepresented settings. As such, we introduce OpenBibleTTS, which is a large-scale benchmark for low-resource speech synthesis spanning 37 underrepresented languages. Moreover, a systematic comparison of various TTS architectures and large-scale speech generation models is conducted across in-domain Biblical text and out-of-domain material. Results show that no single system dominates across languages and metrics: Gemini-TTS achieves the highest listener ratings on most evaluated languages, but monolingual EveryVoice models trained on OpenBibleTTS remain strongest for intelligibility and are preferred in several African languages, while open from-scratch systems degrade sharply on out-of-domain text, revealing a persistent gap between broad multilingual coverage and reliable synthesis quality in underserved linguistic communities. We complement automatic evaluation with subjective human judgments, and open-source all processed datasets, alignments, and trained models to support future low-resource TTS research.
2026-06-07
arXiv (preprint)
doi.org
arxiv.org
Perturbative study of Supercritical Crossover in Noncommutative-corrected Spacetime
Ankit Anand
Shoucheng Wang
We analytically study the Widom line and supercritical crossover of noncommutative charged AdS black holes. Treating the noncommutative para… (see more)meter
2026-06-07
arXiv (preprint)
doi.org
arxiv.org
Personalized and Robust Proactive Robot Assistance with Uncertainty-Guided LLM Reasoning
Alvaro Gonzalez
M. H. Hasan Shovo
Ali Ayub
Proactive robot assistance in household environments requires accurate prediction of human activities and object usage under dynamic and noi… (see more)sy conditions. Existing approaches often rely on complex spatio-temporal models, which can be computationally expensive and sensitive to environmental variability. In this paper, we propose GLOBE, a lightweight framework that combines n-gram Markov models for capturing temporal behavioral patterns with uncertainty-guided large language model (LLM) reasoning. The framework performs sequential prediction efficiently while selectively invoking LLM reasoning only when the model confidence is low. To evaluate performance under realistic conditions, we introduce HOMER-Noise, a noisy extension of the HOMER+ dataset that simulates structured disturbances such as object movements caused by humans, pets, and toddlers. Experimental results show that GLOBE achieves competitive performance with state-of-the-art methods while improving robustness and computational efficiency across both clean and noisy settings. The framework is further validated through a proof-of-concept integration with a Stretch 3 mobile manipulator, demonstrating its potential application in real-world human-robot interaction scenarios.
2026-06-06
arXiv (preprint)
doi.org
arxiv.org
Scaling Decision-Focused Learning to Large Problems with Lagrangian Decomposition
Stéphane Eilles-Chan Way
Hugo Percot
Quentin Cappart
Tias Guns
Louis-Martin Rousseau
Decision-focused learning has shown great promise for addressing predict-then-optimize problems, particularly in the presence of under-speci… (see more)fied models. However, its practical deployment is often hindered by high computational costs and limited scalability, as it requires solving a constrained optimization problem for each training instance at every iteration. To address these challenges, we propose a novel framework that incorporates Lagrangian decomposition into the decision-focused learning paradigm. Specifically, we introduce a new surrogate objective along with two loss functions for evaluating and training the underlying prediction model. We further propose two variants of our approach, which offer different trade-offs between computational efficiency and solution quality. Our framework can be seamlessly integrated with standard decision-focused learning methods, including Smart Predict-then-Optimize (SPO+) and Implicit Maximum Likelihood Estimation (IMLE). Through experiments on two standard benchmarks, the multi-dimensional knapsack problem and quadratic portfolio optimization, we demonstrate that our approach achieves competitive performance while remaining amenable to parallelization. In particular, it consistently outperforms traditional decision-focused learning methods on large-scale instances, involving up to eight times more variables than those typically considered in related work. The implementation is available at https://github.com/corail-research/DFL-LD.
2026-06-06
arXiv (preprint)
doi.org
arxiv.org
IDP-Bench: Benchmarking ability of LLMs to protect personal information in interdependent privacy contexts
Ayana Hussain
Soumya Sharma
Golnoosh Farnadi
Nicholas Vincent
Héber Hwang Arcolezi
Ulrich Aivodji
Large language models (LLMs) are becoming widely deployed as personal AI assistants with access to sensitive user data, making privacy a maj… (see more)or challenge for their design and evaluation. Prior work focuses mainly on individual-level risks, overlooking \textbf{interdependent privacy (IDP)}--where one person's data may be revealed by others without their knowledge or consent. We address this gap by introducing \textbf{IDP-Bench}: the first LLM benchmark for IDP scenarios, grounded in the Contextual Integrity (CI) framework. We evaluate eight open-source LLMs on their understanding of IDP scenarios across three levels of IDP reasoning using two LLM judges. Results show strong co-ownership recognition (6/8 models exceed 90\%) but persistent weaknesses in identifying CI parameters (information attribute, primary subject) and IDP-specific parameters such as secondary subjects, where 7/8 models score below 74\%. Models also struggle to judge sharing appropriateness (5/8 scoring below 77\%). While the ability to judge the appropriateness of sharing improves with scale, performance tends to decline in smaller models, and prompt sensitivity remains high on IDP-specific questions--highlighting the need for more targeted study of IDP in LLM privacy research. Data \& code available \href{https://github.com/tisl-lab/Interdependent_Privacy_Bench}{here}.
2026-06-05
arXiv (preprint)
doi.org
arxiv.org
Prognostic data extraction harnessing a privacy-preserving large language model: a clinician-AI collaborative retrospective evaluation in head and neck oncology
Yujing Zou
Laya Rafiee Sevyeri
Farhood Farahnak
George Shenouda
Marie Duclos
Tomás Yokoo Teodoro de Souza
Khalil Sultanem
Parsa Bagherzadeh
Farhad Maleki
Shirin A. Enger
Privacy regulations and limited expert-validation constrain the deployment of large language models (LLMs) for electronic health record stru… (see more)cturing. We evaluated locally deployed LLMs to extract 30 prognostic variables from 1,360 head and neck cancer reports (882 patients) using zero-shot prompting. A stratified 50-case subset was reviewed by three radiation oncologists (50 cases, 30 fields, 3 reviewers; 4,500 decisions) to form a majority-vote reference for Llama3.3-70B, which achieved 98.6% F1 with high clinician agreement and processed reports in 53 s/report. Among seven additional models (2.6B-70B) benchmarked against this reference, GPT-OSS-20.9B (F1 89.4%) and MedGemma-27B (F1 88.5%) performed best. Integrating LLM-extracted HPV status, smoking history, and Charlson Comorbidity Score into a multivariate Cox Proportional Hazards model (age, sex, T/N stage) improved disease-free survival (likelihood ratio test p = 0.014; ΔC-index + 0.071) and locoregional failure-free survival (p = 0.026; ΔC-index + 0.108) with 1,000-bootstrap internal validation. This clinician-AI collaborative evaluation shows that on-premises LLMs enable privacy-preserving and efficient tumour board support, longitudinal data curation, and outcome prediction.
2026-06-05
npj Precision Oncology (published)
doi.org
Rules of the game: Legislative exits in four Westminster systems
Alex B. Rivard
Marc André Bodet
Jean-François Godbout
By leveraging over 150 years of electoral and biographical data in the Canadian provinces of Ontario, Quebec, New Brunswick, and Nova Scot… (see more)ia, we argue that voluntary exit is best understood as a cost-benefit calculation shaped by positional and institutional incentives in the legislative arena. We show that institutional changes that make seeking re-election costlier are associated with an increased likelihood of a legislator voluntarily exiting the legislative arena. We also find that the determinants of exit vary across age cohorts: younger legislators are more sensitive to institutional and positional cost-benefit incentives, reflecting greater professional mobility and outside career opportunities. Overall, our results indicate that positional and institutional in part explain a legislator’s decision to not seek re-election, but that their impact of these incentives is mediated by life-cycle and retirement-horizon considerations.
2026-06-05
International Political Science Review (published)
doi.org
Shared Latent Structures Enable Unified Backdoor Detection and Mitigation in LLMs
Omar Mahmoud
Aly M. Kassem
Thommen George Karimpanal
Buddhika Laknath Semage
Negar Rostamzadeh
Golnoosh Farnadi
Santu Rana
Backdoor attacks in large language models (LLMs) are often treated as isolated trigger-response failures, motivating defenses tailored to sp… (see more)ecific triggers or behaviors. We show this view is incomplete. Across diverse backdoor behaviors, we identify a shared latent mechanism that can be detected, causally controlled, and suppressed. Using sparse autoencoders (SAEs) on residual-stream activations, we find a small set of latent features consistently activated across jailbreaking, refusal manipulation, password-locking, bias induction, sentiment misclassification, and country-conditioned harmful advice. These features generalize across Qwen3, Gemma~3, and Llama~3.1 models from 4B to 32B parameters, and across both fine-tuning and weight-editing attacks. Through bidirectional activation steering, we show these features are causal: suppressing them reduces attack success, while amplifying them induces target behaviors on clean prompts. We further train lightweight SAE-feature classifiers that generalize zero-shot to unseen backdoors and outperform residual-stream and weight-diffing baselines. Finally, we introduce Concept Ablation Fine-Tuning (CAFT), which suppresses backdoor formation by ablating the shared latent subspace during training. Together, our results suggest that many backdoors rely on a transferable latent mechanism, enabling unified detection and mitigation.
2026-06-05
arXiv (preprint)
doi.org
arxiv.org
SKILL.nb: Selective Formalization and Gated Execution for Durable Agent Workflows
Amine El hattami
Nicolas Chapados
Christopher Pal
AI agents increasingly turn past experience into reusable artifacts such as code, workflows, and procedural memories. Reuse can improve effi… (see more)ciency, but it also creates a lifecycle reliability problem: artifacts that succeed once may fail under environment drift, underspecified tasks, or changing task distributions, especially in web automation. We introduce SKILL.nb, a framework for governing reusable agent workflows with evidence-calibrated lifecycle policies. SKILL.nb uses selective formalization: execution evidence decides which workflow steps should become executable code, which should remain natural-language guided, and when those choices should be revised. Workflows are stored as auditable, versioned notebooks that interleave natural-language guidance, multi-language executable cells, validation gates, fallback paths, and multimodal evidence such as outputs, screenshots, and error traces. At runtime, gate-conditioned execution lets each step run code when its gates validate, or fall back locally when drift invalidates the executable realization. On WebArena-Verified, SKILL.nb achieves 53.7% single-round success, improving over the strongest baseline by 3.9 percentage points. Across three re-executions, it retains 91.7% of initially successful tasks, 15.5 points above the next best method. Under bounded repair, it recovers 72.9% of subsequent failures while limiting post-repair regressions to 4.2%, compared with 15.0% to 17.0% for persistent baselines. It also leads on Mind2Web cross-website and cross-domain splits. In a GitLab migration test, SKILL.nb preserves performance when reusing frozen state learned on GitLab 15.7, with frozen-versus-fresh target-version gaps of -1.7 points on GitLab 16.11 and +0.6 points on GitLab 18.9. These results identify lifecycle governance and gate-conditioned execution as reliability axes beyond one-shot task success.
2026-06-05
arXiv (preprint)
doi.org
arxiv.org