Publications

Does learning the right latent variables necessarily improve in-context learning?
Sarthak Mittal
Eric Elmoznino
Leo Gagnon
Sangnie Bhardwaj
Guillaume Lajoie
Dhanya Sridhar
Large autoregressive models like Transformers can solve tasks through in-context learning (ICL) without learning new weights, suggesting ave… (see more)nues for efficiently solving new tasks. For many tasks, e.g., linear regression, the data factorizes: examples are independent given a task latent that generates the data, e.g., linear coefficients. While an optimal predictor leverages this factorization by inferring task latents, it is unclear if Transformers implicitly do so or if they instead exploit heuristics and statistical shortcuts enabled by attention layers. Both scenarios have inspired active ongoing work. In this paper, we systematically investigate the effect of explicitly inferring task latents. We minimally modify the Transformer architecture with a bottleneck designed to prevent shortcuts in favor of more structured solutions, and then compare performance against standard Transformers across various ICL tasks. Contrary to intuition and some recent works, we find little discernible difference between the two; biasing towards task-relevant latent variables does not lead to better out-of-distribution performance, in general. Curiously, we find that while the bottleneck effectively learns to extract latent task variables from context, downstream processing struggles to utilize them for robust prediction. Our study highlights the intrinsic limitations of Transformers in achieving structured ICL solutions that generalize, and shows that while inferring the right latents aids interpretability, it is not sufficient to alleviate this problem.
2025-04-30
International Conference on Machine Learning (poster)
doi.org
proceedings.mlr.press
AI for Global Climate Cooperation: Modeling Global Climate Negotiations, Agreements, and Long-Term Cooperation in RICE-N
Tianyu Zhang
Andrew Williams
Phillip Wozny
Kai-Hendrik Cohrs
Koen Ponse
Marco Jiralerspong
Soham Phade
Sunil Srinivasa
Lu Li
Yang Zhang
Prateek Gupta
Erman Acar
Irina Rish
Yoshua Bengio
Stephan Zheng
Comprehensive global cooperation is essential to limit global temperature increases while continuing economic development, e.g., reducing se… (see more)vere inequality or achieving long-term economic growth. Achieving long-term cooperation on climate change mitigation with n strategic agents poses a complex game-theoretic problem. For example, agents may negotiate and reach climate agreements, but there is no central authority to enforce adherence to those agreements. Hence, it is critical to design negotiation and agreement frameworks that foster cooperation, allow all agents to meet their individual policy objectives, and incentivize long-term adherence. This is an interdisciplinary challenge that calls for collaboration between researchers in machine learning, economics, climate science, law, policy, ethics, and other fields. In particular, we argue that machine learning is a critical tool to address the complexity of this domain. To facilitate this research, here we introduce RICE-N, a multi-region integrated assessment model that simulates the global climate and economy, and which can be used to design and evaluate the strategic outcomes for different negotiation and agreement frameworks. We also describe how to use multi-agent reinforcement learning to train rational agents using RICE-N. This framework underpinsAI for Global Climate Cooperation, a working group collaboration and competition on climate negotiation and agreement design. Here, we invite the scientific community to design and evaluate their solutions using RICE-N, machine learning, economic intuition, and other domain knowledge. More information can be found on www.ai4climatecoop.org.
2025-04-30
International Conference on Machine Learning (poster)
doi.org
proceedings.mlr.press
Generative AI: Hype, Hope, and Responsible Use in Science and Everyday Life
Doina Precup
2025-04-30
Biological Psychiatry (published)
doi.org
Impact of through‐slice gradient optimization for dynamic slice‐wise shimming in the cervico‐thoracic spinal cord
Arnaud Breheret
Alexandre D'Astous
Yixin Ma
Jason P. Stockmann
Julien Cohen‐Adad
This study investigates the effectiveness of through‐slice gradient optimization in dynamic slice‐wise B0 shimming of the cervico‐thor… (see more)acic spinal cord to enhance signal recovery in gradient‐echo (GRE) EPI sequences commonly used in functional MRI studies. Six volunteers underwent MRI acquisitions with dynamic shim updating (DSU) using a custom‐built 15‐channel AC/DC coil at 3 T. A magnetization‐prepared rapid gradient echo was acquired to segment the spine and to provide a clear image of the anatomical region of interest in the figures. GRE B0 field maps were used to measure field homogeneity before and after shimming; the pre‐shimming field map was used for optimization. Shimmed fields were dynamically applied to GRE–echo planar imaging acquisitions simulating functional MRI acquisitions under two shimming conditions: DSU with and without through‐slice gradient consideration. DSU with through‐slice gradient optimization increased the temporal signal‐to‐noise ratio at the T2 vertebral level by 201% compared with volume‐wise shim and by 28% compared with DSU without through‐slice. The residual geometric distortions were similar between DSU with and without through‐slice gradient optimization. A high signal loss penalty parameter was effective in simulations for reducing through‐slice gradient‐induced signal loss but led to instability and reduced image quality in actual acquisitions due to excessive in‐plane B0 inhomogeneities. Introducing a carefully balanced through‐slice gradient parameter in slice‐wise shimming substantially improves signal recovery in axial GRE images of the spinal cord, without compromising in‐plane homogeneity. This effective approach can advance spinal cord functional MRI applications at high field strengths.
2025-04-30
Magnetic Resonance in Medicine (published)
doi.org
Mitigating Plasticity Loss in Continual Reinforcement Learning by Reducing Churn
Hongyao Tang
Johan Obando-Ceron
Pablo Samuel Castro
Aaron Courville
Glen Berseth
Plasticity, or the ability of an agent to adapt to new tasks, environments, or distributions, is crucial for continual learning. In this pap… (see more)er, we study the loss of plasticity in deep continual RL from the lens of churn: network output variability for out-of-batch data induced by mini-batch training. We demonstrate that (1) the loss of plasticity is accompanied by the exacerbation of churn due to the gradual rank decrease of the Neural Tangent Kernel (NTK) matrix; (2) reducing churn helps prevent rank collapse and adjusts the step size of regular RL gradients adaptively. Moreover, we introduce Continual Churn Approximated Reduction (C-CHAIN) and demonstrate it improves learning performance and outperforms baselines in a diverse range of continual learning environments on OpenAI Gym Control, ProcGen, DeepMind Control Suite, and MinAtar benchmarks.
2025-04-30
International Conference on Machine Learning (poster)
doi.org
proceedings.mlr.press
Multi-Modal Language Models as Text-to-Image Model Evaluators
Jiahui Chen
Candace Ross
Reyhane Askari Hemmat
Koustuv Sinha
Melissa Hall
Michal Drozdzal
Adriana Romero
2025-04-30
ArXiv (preprint)
doi.org
arxiv.org
Outsourced Diffusion Sampling: Efficient Posterior Inference in Latent Spaces of Generative Models
Siddarth Venkatraman
Mohsin Hasan
Minsu Kim
Luca Scimeca
Marcin Sendera
Yoshua Bengio
Glen Berseth
Nikolay Malkin
Any well-behaved generative model over a variable …
2025-04-30
International Conference on Machine Learning (poster)
doi.org
proceedings.mlr.press
Real-time fine finger motion decoding for transradial amputees with surface electromyography
Zihan Weng
Yang Xiao
Peiyang Li
Chanlin Yi
Pouya Bashivan
Hailin Ma
Guang Yao
Yuan Lin
Fali Li
Dezhong Yao 0001
Jingming Hou
Yangsong Zhang
Peng Xu
2025-04-30
Neural Networks (published)
doi.org
Replication of a GWAS signal near HLA-DQA2 with AML using a disease-only cohort and external population-based controls
Rose Laflamme
Véronique Lisi
Josée Hébert
Guy Sauvageau
Sébastien Lemieux
Vincent-Philippe Lavallee
Guillaume Lettre
2025-04-30
Blood Neoplasia (published)
doi.org
On the generalization of language models from in-context learning and finetuning: a controlled study
Andrew Lampinen
Arslan Chaudhry
Stephanie C.Y. Chan
Cody Wild
Diane Wan
Alexander Y. Ku
Alex Ku
Jörg Bornschein
Razvan Pascanu
Murray P. Shanahan
James L McClelland
2025-04-30
ArXiv (preprint)
doi.org
arxiv.org
VinePPO: Refining Credit Assignment in RL Training of LLMs
Amirhossein Kazemnejad
Milad Aghajohari
Eva Portelance
Alessandro Sordoni
Siva Reddy
Aaron Courville
Nicolas Le Roux
Large language models (LLMs) are increasingly applied to complex reasoning tasks that require executing several complex steps before receivi… (see more)ng any reward. Properly assigning credit to these steps is essential for enhancing model performance. Proximal Policy Optimization (PPO), a common reinforcement learning (RL) algorithm used for LLM finetuning, employs value networks to tackle credit assignment. However, recent approaches achieve strong results without it, raising questions about the efficacy of value networks in practice. In this work, we systematically evaluate the efficacy of value networks and reveal their significant shortcomings in reasoning-heavy LLM tasks, showing that they often produce poor estimate of expected return and barely outperform a random baseline when comparing alternative steps. This motivates our key question: Can improved credit assignment enhance RL training for LLMs? To address this, we propose VinePPO, a straightforward approach that leverages the flexibility of language environments to compute unbiased Monte Carlo-based estimates. Our method consistently outperforms PPO and other baselines across MATH and GSM8K datasets in less wall-clock time (up to 3.0x). Crucially, it achieves higher test accuracy for a given training accuracy, capturing more generalization signal per sample. These results emphasize the importance of accurate credit assignment in RL training of LLM.
2025-04-30
International Conference on Machine Learning (poster)
doi.org
proceedings.mlr.press
Caffeine induces age-dependent increases in brain complexity and criticality during sleep
Philipp Thölke
Maxine Arcand-Lavigne
Tarek Lajnef
Sonia Frenette
Julie Carrier
Karim Jerbi
Caffeine is the most widely consumed psychoactive stimulant worldwide. Yet important gaps persist in understanding its effects on the brain,… (see more) especially during sleep. We analyzed sleep electroencephalography (EEG) in 40 subjects, contrasting 200 mg of caffeine against a placebo condition, utilizing inferential statistics and machine learning. We found that caffeine ingestion led to an increase in brain complexity, a widespread flattening of the power spectrum’s 1/f-like slope, and a reduction in long-range temporal correlations. Being most prominent during non-rapid eye movement (NREM) sleep, these results suggest that caffeine shifts the brain towards a critical regime and more diverse neural dynamics. Interestingly, this was more pronounced in younger adults (20–27 years) compared to middle-aged participants (41–58 years) during rapid eye movement (REM) sleep, while no significant age effects were observed during NREM. Interpreting these data in the light of modeling and empirical work on EEG-derived measures of excitation-inhibition balance suggests that caffeine promotes a shift in brain dynamics towards increased neural excitation and closer proximity to a critical regime, particularly during NREM sleep.
2025-04-29
Communications Biology (published)
doi.org