Publications

Forecasting Emerges from Auto-Regressive Pretraining: Latent Predictive Structure in Language Models
Alexis Roger
Prateek Humane
Zhenghan Tai
Gwen Legate
Andrei Mircea
Vasilii Feofanov
Irina Rish
Predicting how a sequence will continue is a basic problem for intelligent systems. We show that large language models contain usable foreca… (voir plus)sting structure before any explicit time-series supervision. A single linear readout from frozen Qwen3-0.6B hidden states maps ordinary text sequences to numerical trajectories that resemble real time series, and those trajectories can be used for straightforward forecasts. The distribution over output tokens also gives coherent, non-crossing probabilistic forecasts in a single forward pass. After time-series specialization, pretrained models show aligned gradients and improve immediately, whereas randomly initialized models spend early training in a destructive-interference regime. These findings suggest that auto-regressive pretraining already shapes representations around temporal continuation; and finetuning adapts that structure to numerical forecasting rather than creating it from scratch.
2026-06-10
ICML.cc/2026/Workshop/Forecast (présentation orale)
openreview.net
openreview.net
Hidden-State Similarity Predicts Re-Elicitation After Inoculation Prompting
Rose Nayoung Kwon
Samy Mammeri
Sabyasachi Sahoo
Christian Gagné
Thomas Jiralerspong
Fine-tuning on narrow harmful tasks can cause emergent misalignment, where models generalize harmful behavior beyond the training distributi… (voir plus)on. Inoculation prompting can reduce this effect by explicitly eliciting the undesired behavior during training, but recent work shows that the behavior can reappear when evaluation prompts contain cues from the training context. We study what makes such prompts effective triggers. We find that textual similarity to the inoculation prompt is an incomplete predictor: prompts are more likely to re-elicit suppressed behavior when they induce activation states similar to those produced by the inoculation context. These findings advance our understanding of how inoculation prompting modulates conditional misalignment, and suggest that activation-space analysis can help identify when suppressed behaviors remain accessible under eval-time prompts.
2026-06-10
ICML.cc/2026/Workshop/Mech_Interp (poster)
openreview.net
openreview.net
NeuroFaith: Evaluating Mechanistic Faithfulness of LLM Free Text Self-Explanation at the Concept Level
Milan Bhan
Jean-Noël Vittaut
Nicolas Chesneau
Sarath Chandar
Marie-Jeanne Lesot
Large Language Models (LLMs) can generate plausible free text self-explanations to justify their answers. However, these natural language ex… (voir plus)planations may not accurately reflect the model's actual reasoning process, indicating a lack of faithfulness. Existing faithfulness evaluation methods rely primarily on behavioral tests or computational block analysis without examining the semantic content of internal neural representations. This paper proposes NeuroFaith, a flexible framework that measures the faithfulness of LLM free text self-explanation by identifying key concepts within explanations and mechanistically testing whether these concepts actually influence the model's predictions. We show the versatility of NeuroFaith across 2-hop reasoning and classification tasks. Additionally, we develop a linear faithfulness probe based on NeuroFaith to detect unfaithful self-explanations from representation space and improve faithfulness through steering. NeuroFaith provides a principled approach to evaluating and enhancing the faithfulness of LLM free text self-explanations, addressing critical needs for trustworthy AI systems.
2026-06-10
ICML.cc/2026/Workshop/Mech_Interp (poster)
openreview.net
openreview.net
The Culture Funnel: You Can't Align What isn't in the Data
Ananya Sahu
Mehrnaz Mofakhami
Daniel D'souza
Thomas Euyang
Julia Kreutzer
Marzieh Fadaee
Current cultural alignment approaches focus on inference-time interventions, assuming models already contain sufficient cultural knowledge. … (voir plus)We argue modern LLM pipelines suffer from a cultural data funnel. Using a multidimensional tagging framework across pretraining, fine-tuning, alignment, and reasoning datasets, we show explicit cultural signals decline sharply during post-training, while geographically concentrated, task-specialized data dominates. Multilinguality enhances geographic diversity of cultural knowledge but does not ensure balanced representation. Our tags improve downstream cultural benchmark performance, demonstrating that advances require shifting focus in training data pipelines. To facilitate future research, we release our culturally tagged dataset with 5.6M samples at https://huggingface.co/datasets/CohereLabs/CultureMarkers.
2026-06-10
arXiv (prépublication)
doi.org
arxiv.org
The strength of flow refueling location problem formulations and an extension to cyclic routing
Nagisa Sugishita
Margarida Carvalho
Ribal Atallah
2026-06-10
4OR (publié)
doi.org
WEAVER, Better, Faster, Longer: An Effective World Model for Robotic Manipulation
Arnav Kumar Jain
Yilin Wu
Jesse Farebrother
Gokul Swamy
Andrea Bajcsy
The potential impacts of world models (WMs, i.e., learned simulators) on robotics are far-reaching -- policy evaluation, policy improvement,… (voir plus) and test-time planning -- all with limited real-world interaction. To unlock these downstream capabilities, a WM needs to jointly satisfy three desiderata:
2026-06-10
arXiv (prépublication)
doi.org
arxiv.org
When Does Interleaving Prevent Emergent Misalignment?
Samy Mammeri
Rose Nayoung Kwon
Chen Sun
Sabyasachi Sahoo
Christian Gagné
Thomas Jiralerspong
Large language models finetuned on narrow harmful tasks are prone to emergent misalignment (EM), where harmful behavior generalizes beyond t… (voir plus)he training distribution. Interleaving benign data during finetuning has been proposed as a mitigation, but recent work disagrees on whether it prevents EM. In this paper, we investigate this disagreement on Qwen-2.5 7B and 32B, and find that no single property of the interleaved data, taken in isolation, accounts for the gap. Instead, much of it traces to the evaluation itself, as the standard EM benchmark is sensitive to the length of the prompts it uses, and lengthening the evaluation prompts substantially shifts measured misalignment across model sizes. We then identify a region in the model's activations that predicts whether a given interleaved set will prevent EM, and show that reformulating benign data to fall within it substantially reduces EM on both 7B and 32B. This suggests that the standard EM benchmark, which relies on short prompts, may misrepresent the effectiveness of proposed mitigations.
2026-06-10
ICML.cc/2026/Workshop/Mech_Interp (poster)
openreview.net
openreview.net
AfriSUD: A Dependency Treebank Collection for Evaluating Models on African Languages
Happy Buzaaba
Cheikh Mouhamadou Bamba Dione
David Ifeoluwa Adelani
Sylvain Kahane
Kim Gerdes
Bruno Guillaume
Kevin Guan
Aremu Anuoluwapo
Naome A. Etori
Shamsuddeen Hassan Muhammad
Utitofon Inyang
Peter Nabende
David Sabiiti Bamutura
Andiswa Bukula
Chinedu Uchechukwu
Rooweither Mabuya
Idris Akinade
Christiane Fellbaum
Despite their linguistic diversity and global significance, African languages remain underrepresented in research and resources to support N… (voir plus)LP. We aim to bridge this gap by introducing AfriSUD, the first large-scale collection of syntactically annotated treebanks for nine diverse African languages spanning major language families and regions across Sub-Saharan Africa. Using the Surface-Syntactic Universal Dependencies (SUD) framework, our community-led effort provides high-quality, native-speaker verified data that capture typological key features such as agglutination and tone. We evaluate a range of models on AfriSUD for part-of-speech tagging and dependency parsing including non-transformer baselines, multilingual pretrained encoders, and LLMs. Our results reveal a significant syntax gap, where models still show clear limitations across the nine languages, suggesting that existing architectures may not fully capture the structural diversity of African-language syntax.
2026-06-09
arXiv (prépublication)
doi.org
arxiv.org
Dynamic Neural Graph Encoding of Inference Processes in Deep Weight Space
Di Wu
Huan Liu
Zhixiang Chi
Yuanhao Yu
Konstantinos N. Plataniotis
Yang Wang
The rapid advancements in using neural networks as implicit data representations have attracted significant interest in developing machine l… (voir plus)earning methods that analyze and process the weight spaces of other neural networks. However, efficiently handling these high-dimensional weight spaces remains challenging. Existing methods often overlook the sequential nature of layer-by-layer processing in neural network inference. In this work, we propose a novel approach using dynamic graphs to represent neural network parameters, capturing the temporal dynamics of inference. Our Dynamic Neural Graph Encoder (DNG-Encoder) processes these graphs, preserving the sequential nature of neural processing. Additionally, we also leverage DNG-Encoder to develop INR2JLS (Implicit Neural Representation to Joint Latent Space) for facilitate downstream applications, such as classifying Implicit Neural Representations (INRs). Our approach demonstrates significant improvements across multiple tasks, surpassing the state-of-the-art INR classification accuracy by approximately 10\% on the CIFAR-100-INR. Our code is available at https://github.com/dddiowww/DNG.
2026-06-09
Transactions on Machine Learning Research (accepté)
openreview.net
openreview.net
Establishment of a tissue culture system with adventitious bud regeneration for the new raspberry germplasm 'autumn–winter yellow raspberry'
Zihan Zhu
Hanqing Zhao
Jinyu Liu
Ye Guo
Chenxing Zhang
Yingyue Li
2026-06-09
Plant (publié)
doi.org
Hierarchical Integration of Predictive Representations of State from General Value Functions
Sonny Jones
Doina Precup
Patrick M. Pilarski
Ashley N Dalrymple
In this work, we investigate how predictive representations of state in the form of continually learned General Value Functions (GVFs) inter… (voir plus)act with downstream policy networks. Intelligent agents deployed in real-world environments need to adapt to changing conditions in their environment. Adapting to one’s environment requires a model or representation of the environment on which to base decision-making. Models that take the form of predictions and GVFs have been shown to provide temporally abstracted predictive representations of state that can forecast useful elements of an agent's or environment's future behaviour. While GVFs have been concretely deployed in rehabilitation and robotic domains, existing approaches treat predictions as input features into model frameworks, without examining or comparing how best to integrate them into downstream learning processes. In this work, we compare multiple strategies for integrating observations and GVF predictions into another learning architecture: 1) actual observations solely in the input layer, 2) predictions solely in the input layer, 3) actual observations and predictions in the input layer, and 4) actual observations in the input layer and predictions in the later latent representations. We evaluate these strategies in a rehabilitation setting, using GVFs to learn predictive representations of kinetic and kinematic signals collected from wearable sensors on the lower limb during ambulation across varied terrains, and policy networks to classify walking terrain.
2026-06-09
rl-conference.cc/RLC/2026/Workshop/RL_in_Big_Worlds (poster)
openreview.net
openreview.net
Learning by Chatting? Investigating the Impact of Generative AI on Information Seeking and Learning
Shravika Mittal
Su Lin Blodgett
Q. Vera Liao
Generative AI (GenAI) tools offer increasing opportunities for augmenting human cognitive tasks. Among these tasks, information seeking is b… (voir plus)eing rapidly reshaped by GenAI tools, with potentially profound implications for learning and knowledge acquisition. To investigate these implications, we conducted a between-subjects field experiment in which participants pursued informal learning by seeking information through either ChatGPT or Google Search over a span of 8 days. Using a daily diary protocol, we gathered in-situ data on their information-seeking processes. Our findings show that participants in the ChatGPT group experienced diminished agency in their information-seeking processes, as they offloaded much of the information selection to AI, and consequently experienced greater meta-cognitive load arising from this reduced sense of control. We further highlight two sources of distortion in information access when using ChatGPT: biases in ChatGPT outputs, particularly towards providing solution-oriented artifacts over principled knowledge; and systematic shifts in users' information-seeking behaviors, whereby the conversational and socially-oriented interaction paradigm of current GenAI tools may inadvertently reduce exploration of the broader knowledge space. As a result, on average, participants in the ChatGPT group had worse learning outcomes than those using Google, especially for higher-order critical learning. Our work suggests inherent tensions between offloading information seeking to AI and meaningful learning, and provides broader implications for understanding AI's risks to human cognition.
2026-06-09
arXiv (prépublication)
doi.org
arxiv.org