Publications

Probe to Generate: Program Variant-Guided Test Augmentation for Repository-Level Repair Benchmarks
Zehao Wang
Shin Hwei Tan
Revisiting Explainable AI through Model-Independent Concept Dictionaries
Thomas Schnake
Doreen Schöppenthau
Alexander Meyer
Klaus-Robert Müller
Grégoire Montavon
Modern applications of AI rely on increasingly complex models. Explainable AI (XAI) has emerged as a set of techniques aimed at improving mo… (voir plus)del transparency. However, existing XAI methods typically assume input features to be inherently interpretable, or they rely on intermediate internal abstractions that are difficult to characterize and highly architecture-specific, hindering consistent use across models. To address these limitations, we propose DictXAI, a method that defines concepts directly in the input domain via a dictionary---a large, potentially overcomplete set of predefined elements, each carrying an interpretable meaning. Technically, DictXAI first computes a sparse code of the input and then attributes the model's prediction to the associated dictionary elements. We demonstrate the actionable nature of DictXAI explanations, showing that they can attribute AI malfunctions (e.g., Clever Hans effects) directly to identifiable artifact patterns in the data, while fostering human-AI alignment on intricate biomedical signals. We further demonstrate our method's ability to operate across a wide variety of dictionaries, including learned image bases, analytically defined waveforms for electrocardiography, and experimentally acquired dictionary elements. Overall, our results show that DictXAI provides more interpretable, actionable, and architecture-agnostic insights than classical XAI or existing concept-based approaches.
RoboJEPA: Scaling Robotic Latent World Models
Nicolas Beltran-Velez
Jianhao Yuan
Tushar Nagarajan
Daniel Severo
Michal Drozdzal
Jeannette Bohg
Mahmoud Assran
Latent world models have shown a remarkable ability to predict future states and to plan in the real world. In practice, however, we lack a … (voir plus)principled way to estimate how their capabilities scale with model size, data, and compute, an open problem that slows progress in the field. In this work we present RoboJEPA, a world model based on the Joint Embedding Predictive Architecture (JEPA) and trained on a large-scale dataset spanning 12 robotic embodiments. We show that RoboJEPA's imagination error, the error of its latent rollouts, follows a second-order power law in compute, allowing us to predict model quality well beyond the scale at which the law is fit. We further show that downstream robotic planning performance improves predictably with compute, and that imagination error is strongly correlated with it, making it a reliable proxy for real-robot evaluation. Finally, we demonstrate that latent world models can be deployed zero-shot as robotic agents, planning toward a single goal image to solve tasks requiring long-horizon planning on real hardware. We release all model checkpoints together with our training and robot deployment code. To our knowledge, this is the first work to establish scaling laws for multi-embodiment robotic world models trained on real robot data, and RoboJEPA, at 8B parameters, is the largest JEPA predictor model trained to date.
Learning Grasp Targeting from Point Clouds for Log Pile Clearing on a Hydraulic Crane
George Sideris
Lucas Bessai
Heshan Fernando
Elie Ayoub
Nicolas Lemieux
In mill yards, log loaders clear dense piles by a sequence of bundle grasps: hundreds of logs rest in contact, and each removal changes the … (voir plus)pile available to the next grasp. A learned policy chooses where to place and orient the grapple from unsegmented point clouds and runs on a trailer-mounted hydraulic forestry crane. The policy classifies at which observed point to grasp and predicts depth and grapple orientation there. The same network outputs support behavior cloning (BC), reinforcement learning (RL), and deployment. BC learns from successful top-of-pile demonstrations; RL explores for improvements by fine-tuning the cloned policy (BC
Quadratic Weak-to-Strong Generalization in Random Feature Networks via Random Matrix Theory
Deborah Oliveira
Weak-to-strong generalization is the phenomenon where a strong student model trained with labels produced by a weak teacher model is able to… (voir plus) generalize better than the teacher. In this paper, we study this phenomenon in two-layer random feature networks where the model strength is determined by its width. Using tools from random matrix theory, we derive deterministic equivalents for the population errors of an optimally trained teacher and a student trained with gradient flow. For ReLU activation and a pure spherical harmonic target, we obtain sharp asymptotics under a Gaussian universality assumption, showing a quadratic improvement: the student error scales as the square of the teacher error. These results attain the general lower bound of Medvedev at al (2025). We also analyze how the student behaves under more general stopping times and targets supported on multiple harmonic degrees, characterizing the regimes in which weak-to-strong generalization occurs and identifying the transition between quadratic, non-quadratic, and no improvement.
A VAR(1)-Regularized Spatial Pattern Regression for Temporally Coherent Reconstruction of High-Resolution Meteorological Fields
Vihotogbé Houssou
Visual Memory Attacks Can Persist Through The KV Cache
Modern language model systems operate autonomously over increasingly long contexts containing untrusted text and images. Can an adversarial … (voir plus)input continue to steer a model even after that input is removed from its context? We show that attacks can be trained to persist through the key/value (KV) cache of subsequent tokens, allowing adversarial influence to outlive direct access to its source.We consider the Visual Memory Injection (VMI; Schlarmann and Hein, 2026) attack setting, in which an adversarial image that stays in the context plants a hidden backdoor: the model behaves normally until a chosen trigger elicits an attacker-chosen response. We first demonstrate persistence in this setting with optimized soft prompts, which remain effective after we mask the prompt from attention. We then introduce Persistent Visual Memory Injection (P-VMI), which optimizes images to preserve this adversarial behaviour after they are masked from attention. These attacks persist over conversations substantially longer than those used during optimization. On Qwen3-VL-8B-Instruct, P-VMI achieves up to approximately
Ancient DNA challenges to admixture narratives: Insights from Notre-Dame Cemetery (1683–1796) in Colonial Montreal, Canada
Diane Martin‐Moya
Jean‐Christophe Grenier
Liam Thomas Lanigan
Justin Pelletier
Evan K. Irving-Pease
Fernando Racimo
Claude Bhérer
Hannes Schroeder
Emmanuel Milot
Isabelle Ribot
Historical narratives of early colonization in Quebec frequently portray French-Indigenous relations as cooperative and characterized by adm… (voir plus)ixture; which are often mobilized in modern debates on false-indigeneity. These narratives have been studied through genealogical, genetic, and bioarchaeological analyses, yet individuals of African descent remain mostly absent from them, while Indigenous peoples are framed largely in relation to colonists. Notre-Dame Cemetery (1683-1796) in Tiohtià:ke-Montréal offers a unique window onto one of the earliest urban colonial populations in Nouvelle France, but previous bioarchaeological studies have produced conflicting interpretations regarding the presence of non-European or admixed individuals. Here, we generated ancient DNA from 45 individuals and applied genotype imputation, runs-of-homozygosity, and identity-by-descent analyses to reevaluate genetic relatedness and cemetery organization. Our results show exclusively French European ancestry. We observe low levels of close-kin unions and identify several genetic lineages distributed across the cemetery's successive burial phases rather than family clustering or religious ordering. The spatial organization of the graves is consistent with rapid demographic growth, with successive burials overlapping one another, before being disrupted by urban development. Rooted in Indigenous decolonial frameworks, these findings challenge the 'admixed founding population' narrative, aligning with historical evidence of Indigenous and African-descended marginalization while highlighting the social inequalities shaping this colonial urban cemetery. Québec's bioarchaeological practices must move beyond blood ties as the sole measure of kinship and commit to legal frameworks that better reflect reciprocal obligations of care, in order to overcome the colonial structures that still define heritage.
Balancing Memory Pathways: Analyzing and Improving Memory Utilization in Hybrid LMs
Hyunji Lee
Joykirat Singh
Zaid Khan
Justin Chih-Yao Chen
Elias Stengel-Eskin
Arman Cohan
Mohit Bansal
Recurrent-attention hybrid language models (LMs), which interleave attention and recurrent layers, are increasingly used to combine the effi… (voir plus)ciency of the recurrent layers with the strong performance of attention layers. Prior work suggests that attention and recurrent layers offer complementary pathways to use past information: attention supports precise memory recall from earlier tokens, while recurrent layers support consolidation of disparate information over long contexts. However, we observe that simply having access to both pathways does not mean that hybrid LMs are effectively using them. We find that they rely substantially more on attention than on the recurrent state. Standard supervised fine-tuning improves overall performance but does not improve how the two memory pathways are coordinated: the model becomes more reliant on information propagated by attention layers, while its use of information propagated by recurrent layers remains limited. To encourage better coordination between the two memory pathways, we add an auxiliary loss that limits attention's access to earlier context while the recurrent state propagates through the full sequence. This objective encourages the model to retain and use information through the recurrent pathway alongside attention. It improves overall performance, with particularly strong gains on tasks involving longer contexts or requiring information aggregation, consistent with the strengths of recurrent layers observed in analysis. Crucially, this imbalance and the benefit of our auxiliary loss generalize: they apply to multiple recurrent-attention LMs in question-answering and agentic tasks, as well as to attention-based LMs that combine different forms of memory. Together, our findings show that simply providing multiple memory pathways does not ensure their effective use, and that targeted supervision is needed to better coordinate them.
A Framework for Context Enrichment and Selective Robustness in LLM Social Agents
Researchers use large language models as agents in social simulations, where agents are prompted with observations and background informatio… (voir plus)n intended to shape their actions. Yet LLM agents have been shown to substantially change their behavior under meaning-preserving prompt variations, creating a validation problem for social simulation. One plausible hypothesis is that this brittleness reflects lack of grounding and can be reduced by enriching the context provided to the agents. Whether richer context improves robustness remains unclear, in part because context richness and robustness are rarely defined in ways that support controlled comparisons. We address the first need with controlled context enrichment, a set of prompt substitution rules that systematically enrich distinct types of context: neutral padding, generic social background, descriptive decision grounding, and normative decision grounding. We address the second by introducing selective robustness as a validation criterion: insensitivity to meaning-preserving prompt variation combined with a directional response to decision-relevant changes in the social world. We demonstrate the framework in a Prisoner's Dilemma-like game, showing how it can be used to identify which forms of context enrichment produce more robust agent behavior. We find that these forms of enrichment produce distinct behavioral patterns that vary between the two models tested.
Fixed-Point Rates for Self-Distillation of Flow Maps
Stanic Tom
Avishek Joey Bose
Flow and diffusion models generate high-quality samples, but inference typically requires many sequential neural-network evaluations. Flow-m… (voir plus)ap models address this bottleneck by learning finite-time transitions directly, enabling generation in one or a few evaluations. Previous work introduced Lagrangian, Eulerian, and progressive self-distillation rules with training targets derived from the model's own predictions. Does reconstructing a candidate flow map from its own predictions correct errors or reinforce them? With the learned velocity field held fixed, we view each rule as an operator on candidate flow maps and study its iterates. By composing two half-length transitions, progressive reconstruction converges globally and geometrically in a weighted norm. This requires spatial Lipschitz bounds and initial errors relative to the teacher flow that vanish uniformly at least quadratically as transitions become shorter. Every reconstruction reduces the worst-case weighted error by at least half, although it can amplify error in the ordinary supremum norm. Lagrangian reconstruction has a factorial convergence guarantee, whereas Eulerian reconstruction admits no general same-order Lipschitz stability bound. Under the stated assumptions, Lagrangian and progressive reconstruction yield two-sided bounds on the error to the frozen teacher flow in terms of the full weighted residual norm. For progressive reconstruction, analogous mean-square bounds hold over teacher-generated states, with a supremum over time pairs. A two-dimensional experiment illustrates how reconstruction reduces errors over short intervals and how its changes indicate the remaining error. Lagrangian reconstruction gives faster initial improvement in this example.
GitSwarm: Decentralized Compounding Inference
Ankur Samanta
Paras Dahal
Mikhail Plekhanov
Carole-Jean Wu
Scott Yih
Remi Munos
Rob Fergus
Jakob Foerster
Ruslan Salakhutdinov
Sanjeev Arora
Jason Weston
Long-horizon problem solving and scientific research require computation to accumulate across successive attempts. Partial solutions, experi… (voir plus)mental findings, and unsuccessful approaches can inform later work, yet most inference-time computation is organized around individual trajectories or candidates rather than a persistent body of reusable work. We call this paradigm compounding inference: organizing inference-time computation so that intermediate work persists and can be inspected, extended, combined, or challenged by subsequent computation. We instantiate compounding inference in GitSwarm, an asynchronous system where homogeneous agents independently decide how to advance a task while collaborating through structured persistent memory. Agents explore, experiment, verify, refine, and synthesize previous work in a shared, branch-able Git repository. Atomic commits preserve intermediate artifacts, while explicit semantic dependencies record how later contributions build on work across branches. We evaluate GitSwarm on long-horizon problem solving and sustained GPU-backed experimental research. On IMOProofBench-Advanced, GitSwarm solves all 30 problems in one run using GPT-5.5. On ProgramBench, it achieves a