Publications

LLM Persona Unlearning
Kemou Li
Qizhou Wang
Fengpeng Li
Jiantao Zhou
Pre-training equips large language models (LLMs) with a broad repertoire of behavioral patterns associated with roles, styles, values, and g… (voir plus)oals. Post-training teaches conditional enactment and makes a helpful Assistant the default, but it does not erase alternative modes from the weights; explicit prompts can therefore elicit personas that repeatedly shape judgment, language, and action. In open-weight settings, runtime controls can be removed, motivating persona unlearning: a weight-level edit that makes a designated persona difficult to elicit and enact on unseen contexts. We introduce PersonaUnlearnBench, a model-specific paired benchmark spanning six LLMs from three families and five personas, with aligned forget/retain sets, held-out instruction paraphrases, and four-axis evaluation. The benchmark shows that standard unlearning methods cannot reliably erase the target persona without sacrificing meaningful generation or general utility. We therefore propose PaCE, which compares target and desirable responses to the same questions to locate an internal behavior direction, then trains target-prompt states away from the target mode and toward the matched desirable response. Experiments show that PaCE consistently suppresses target personas with high response quality and useful counterpart behavior, at moderate utility cost. These results establish persona unlearning as a distinct behavior-level editing problem and a practical route toward persistent control of latent LLM response policies.
LLS: Learnable Layer Selection for Test-Time Adaptation
Suryansh Ram Menon
Frédéric Precioso
While Test-Time Adaptation (TTA) can recover performance drops caused by distribution shifts without requiring source labels, adapting all m… (voir plus)odel parameters frequently leads to catastrophic forgetting and overfitting to noisy test streams. Consequently, layer selection is critical. Existing methods such as GALA (Sahoo 2025) select layers via a cosine alignment proxy between gradient trajectories and parameter updates, a heuristic decoupled from the actual adaptation objective. We propose Learnable Layer Selection (LLS), a differentiable framework that learns continuous per-layer masks. The key insight of this work is not just to learn the layer selection masks, but also the idea to use gradients on one sample to learn the mask on another sample. By evaluating tentative parameter updates on a different, held-out validation sample, we introduce a novel self-supervised generalization loss. This forces the selection mechanism to learn layer masks that generalize to samples not explicitly optimized by current gradients, rather than overfitting. Evaluated on Full DomainBed (ResNet-18, 3 seeds, 16 environments), LLS achieves competitive accuracy across standard TTA algorithms, yielding the best target-split SHOT accuracy of 64.41 ± 0.96%. Systematic evaluations confirm that this cross-sample approach provides a highly stable mechanism for selective parameter adaptation.
MemCodex: Self-Programming Hierarchical Memory for Language Agents
Agent memory faces heterogeneous access needs: a single-hop question may require one piece of evidence, whereas a multi-hop question must co… (voir plus)mbine evidence from multiple sources. Predefined memory workflows cannot adapt to these varying needs. Recent adaptive methods search or learn over memory components and their compositions, but the design space itself remains predefined. We introduce MemCodex, a self-evolving hierarchical memory system that organizes experience into executable memory programs for summaries, relational knowledge, reusable skills, and latent memory. Open-ended program evolution searches the open design space of layer programs by rewriting how each layer is constructed, indexed, retrieved, and routed, thereby adapting both within-layer implementations and cross-layer composition. At query time, reads traverse the hierarchy from coarse to fine and stop once sufficient evidence is found, descending to the original history when needed. We further develop MemArena, a unified runtime that places heterogeneous data and memory systems behind a common interface. MemCodex improves average task success by 10.1% relative to the strongest adaptive-memory baseline, while using 3.4x fewer context tokens and achieving 2.1x faster inference.
Preemptive LLM Unlearning against Forbidden Capability Acquisition via Gradient Sealing
Kemou Li
Qizhou Wang
Yue Wang
Fengpeng Li
Masashi Sugiyama
Jiantao Zhou
Open-weight LLMs are released not only as fixed products but also as substrates for downstream fine-tuning. This openness, however, creates … (voir plus)legal and ethical risks because users may misuse fine-tuning to instill illicit knowledge or enable hostile operations. Model providers therefore need apre-release defense against such acquisition, motivating the problem of preemptive unlearning. Unlike retrospective unlearning, which removes capabilities already present in a fixed model, preemptive unlearning seeks to prevent their acquisition under unseen attack data and future fine-tuning procedures. Despite its practical importance, this setting remains largely unexplored, presents distinct challenges, and is therefore the central focus of our work. We first verify that existing retrospective methods provide insufficient pre-release protection. Even when forbidden capabilities are suppressed in current outputs, forbidden-domain data can still induce gradients through internal pathways, enabling later acquisition. Motivated by this finding, we propose a gradient-sealing principle that blocks these pathways by pushing relevant pre-activations into the negative region, where ReLU-family activations exhibit zero or near-zero derivatives. Experiments across multiple LLM families demonstrate our stronger resistance to downstream acquisition than retrospective baselines, validating gradient sealing as an effective mechanism for pre-release protection.
RainAtlas: A Multi-Continental Dataset for Precipitation Downscaling
Extreme rainfall events are increasing in intensity and frequency as climate change accelerates. While kilometer-scale precipitation forecas… (voir plus)ts are critical for supporting local decision-making, the limited availability of high-resolution precipitation observations hinders their accuracy, especially in under-resourced regions. Machine learning models are widely used to downscale precipitation data to km-scale, but their application to unseen geographies presents challenges. First, processing raw high-resolution precipitation datasets across regions requires significant engineering and domain expertise. Second, generalization across regions remains difficult. To help overcome these barriers, we release RainAtlas, a large-scale, ML-ready and multi-continental dataset for precipitation downscaling. Covering three continents, RainAtlas harmonizes heterogeneous hourly km-scale observations to a common 2-km grid. Each regional partition contains around 210,000 aligned low- and high-resolution precipitation pairs, respectively from ERA5 reanalysis and direct observations. We benchmark state-of-the-art ML-based downscaling models across RainAtlas using a wide range of metrics. Our evaluation reveals substantial variance in out-of-domain generalization depending on the training regions. This underscores the need for cross-regional, multi-source km-scale evaluation, establishing RainAtlas as a well-positioned benchmark for precipitation downscaling research.
Random Recursive Models
Jama Hussein Mohamud
Recursive models create computational depth through parameter reuse, offering a parameter-efficient alternative to increasing model size. Ho… (voir plus)wever, most recursive models repeatedly apply one learned transformation or a prescribed sequence of transformations, restricting computation to a fixed layer order. We introduce the Random Recursive Model (RRM), which maintains a pool of
Real Faults in Model Context Protocol (MCP) Software: a Comprehensive Taxonomy
Mina Taraghi
Mohammad Mehdi Morovati
The rapid adoption of foundation models has significantly expanded the capabilities of software systems, enabling them to perform complex la… (voir plus)nguage, reasoning, and interaction tasks that were previously difficult to automate. However, this progress has also introduced novel challenges that were largely absent in previous generations of software. In particular, the increasing integration of foundation models with external tools and resources raises new concerns regarding reliability, security, and robustness. The Model Context Protocol (MCP) has recently been proposed to standardize interactions between AI-based software systems, software tools, and external resources. Despite its growing adoption, there remains limited systematic understanding of real-world faults in MCP-based software systems. In this paper, we present the first large-scale taxonomy of faults in MCP servers, comprising five high-level fault categories derived from empirical evidence. To evaluate the completeness and generalizability of this taxonomy, we conduct a survey of MCP practitioners representing diverse professional roles. The results confirm that all MCP-specific fault categories occur in practice and reveal distinct characteristics that differentiate MCP-specific faults from non-MCP faults. Overall, these findings provide actionable guidance for building more robust MCP-based systems through boundary-aware testing, response validation, configuration diagnostics, and reproducible issue reporting.
What Does Layer-Importance Reveal About Transformers and State-Space Models?
Transformers and state-space models (SSMs) are the two dominant families of sequence models, and a central open question is how far the anal… (voir plus)ytical knowledge built for transformers transfers to SSMs. We address this through the lens of layer importance which underpins compression, selective fine-tuning, and interpretability across both families. We decompose layer importance into two distinct notions. \emph{Necessity} captures how much the pretrained model depends on a layer's existing contribution, measured by the loss increase from bypassing it. \emph{Plasticity} captures where the model absorbs new information during fine-tuning, measured by the magnitude of task-specific weight updates. Our analysis reveals that the two families behave fundamentally differently: in every evaluated residual transformer up to
AdaptArena: Evaluating Test-Time Personalization of Web Agents
Large language model (LLM) agents have demonstrated strong performance on complex web navigation tasks, yet they remain brittle in real-worl… (voir plus)d settings where user intentions are underspecified and preferences are heterogeneous. In practice, users rarely provide explicit profiles, requiring agents to infer latent preferences from implicit signals. Despite its importance for deployment, this problem setting is largely underexplored in existing benchmarks. To address this gap, we introduce AdaptArena, a benchmark for evaluating test-time personalization of web agents via implicit preference inference. AdaptArena consists of 480 tasks, featuring both single-preference and double-preference scenarios. Each evaluation task must be solved by retrieving and leveraging the most relevant historical user trajectory that implicitly encodes the target preference. In addition, we introduce AdaptiveAgent, a retrieval-based framework for standardized evaluation of implicit preference inference. Experiments reveal a substantial performance gap: while oracle agents with access to ground-truth preferences achieve an 82.92% success rate, the evaluated LLM agents using our framework reach at most 15.62%. Furthermore, we find that correctly inferring user preferences is necessary but not sufficient for task success, as execution failures in downstream web interactions remain a significant bottleneck even when agents align with the target preference. These findings highlight implicit preference inference and robust action grounding as key challenges for deploying reliable, user-facing web agents. We release our code: https://github.com/McGill-NLP/web-agents-test-time-adaptations
A biological-response compound representation allows chemical perturbation prediction across cell lines
Abstract Accurately predicting cellular responses to drugs remains a challenge with the potential to reduce experimental screening costs and… (voir plus) accelerate drug discovery. Current computational approaches represent compounds through chemical structures, which carry little information regarding their activity within biological systems. We show that gene expression responses, measured in a reference cell line, can instead serve as transferable representations of chemical perturbations. The proposed framework, BioPert, uses biological representations with a small neural network to predict transcriptional delta responses in cell lines of different lineages. Across the Tahoe-100M and LINCS L1000 datasets, BioPert outperforms molecular fingerprints and embeddings learned from chemical structure, which provide limited improvements over random controls. BioPert’s prediction correlations reach 0.79 on Tahoe-100M, corresponding to an improvement of 0.34 over the next-best representation. On LINCS, performance varies with the experimental reproducibility of the test conditions, highlighting the impact of batch effects. Notably, BioPert considerably surpasses predictions obtained by copying the reference response, demonstrating that it captures context-dependent effects. Further analysis of a C32-cobimetinib case shows pathway-level accuracy of the predictions, including when the reference and target responses differ. These results open a new path for chemical perturbation prediction and could ultimately reduce the burden of phenotypic screening.
A Closed Form Depth-Width Law for Neural Collapse in Deep Linear Networks
Pankhil Gawade
Kirsten Fischer
Ewa Szczurek
How depth and width of neural networks shape learned representations is usually studied only after training. We show that, in a tractable fe… (voir plus)ature-learning setting, this dependence can instead be obtained in closed form. Using neural collapse as a probe of representation geometry, we express NC1--NC3 directly in terms of the learned feature kernel, extending an identity known for NC1 to the angular part of NC2 and to NC3 under a ridge readout. For deep linear networks with exchangeable class structure, the resulting kernel equations yield a geometric law, exact within the theory, for the layerwise evolution of within-class variability. At large width, the accumulated compression becomes a function of the depth-to-width ratio
Credible Information Subset Decomposition: An End-to-End Multi-fidelity Learning Model by Modeling Label Information
Sihan Wang
Wenjie Du
In the AI4Chemistry scenario, utilizing heterogeneous data at different fidelity levels is a common and core issue. High-fidelity data is ac… (voir plus)curate but scarce, while low-fidelity data is abundant but biased. Traditional multi-fidelity methods typically identify cross-fidelity biases based on paired samples under different fidelity labels. However, due to the mismatch in dataset input distribution and the complexity of the biases themselves, these methods are difficult to implement in real-world scientific environments. To address this, we propose a trusted information subset decomposition framework that can efficiently utilize multi-fidelity data without requiring paired samples. Multi-fidelity label supervision is decomposed into three complementary subsets: a trusted information subset based on the absolute value of high-fidelity labels; a trusted subset that captures the reliability of the high-fidelity and medium-fidelity label intervals through adaptive constraints; and an ordered trusted subset representing the numerical relationships within the same fidelity level. These subsets are then integrated into a unified end-to-end model, enabling the reasonable utilization of medium- and low-fidelity information. Extensive experiments on various molecular and material property benchmarks demonstrate that our method consistently outperforms state-of-the-art multifidelity and singlefidelity baseline methods, and exhibits good robustness under real-world unpaired multifidelity conditions.