Irina Rish

Biography

Irina Rish is a full professor at the Université de Montréal (UdeM), where she leads the Autonomous AI Lab, and a core academic member of Mila – Quebec Artificial Intelligence Institute.

In addition to holding a Canada Excellence Research Chair (CERC) and a CIFAR Chair, she leads the U.S. Department of Energy’s INCITE project on Scalable Foundation Models on Summit & Frontier supercomputers at the Oak Ridge Leadership Computing Facility. She co-founded and serves as CSO of Nolano.ai.

Rish’s current research interests include neural scaling laws and emergent behaviors (capabilities and alignment) in foundation models, as well as continual learning, out-of-distribution generalization and robustness.

Before joining UdeM in 2019, she was a research scientist at the IBM T.J. Watson Research Center, where she worked on various projects at the intersection of neuroscience and AI, and led the Neuro-AI challenge. She was awarded the IBM Eminence & Excellence Award and IBM Outstanding Innovation Award (2018), IBM Outstanding Technical Achievement Award (2017) and IBM Research Accomplishment Award (2009).

She holds 64 patents and has published 120 research papers, several book chapters, three edited books and a monograph on sparse modeling.

Current Students

Ivan Anokhin

PhD - Université de Montréal

Co-supervisor :

Samira Ebrahimi Kahou

ivan.anokhin@mila.quebec

rishika.bhagwatkar@mila.quebec

Rifat Arefin

PhD - Université de Montréal

Arjun Ashok

PhD - Université de Montréal

Co-supervisor :

Alexandre Drouin

arjun.ashok@mila.quebec

Master's Research - Université de Montréal

PhD - Université de Montréal

ethan.caballero@mila.quebec

Pietro Cagnasso

PhD - Concordia University

Principal supervisor :

pietro.cagnasso@mila.quebec

Mohammad Javad Darvishi Bayazi

Amin Darabi

PhD - Université de Montréal

amin.darabi@mila.quebec

Collaborating researcher - Université de Montréal

mj.darvishi92@gmail.com

Wagner Drew

Master's Research - Concordia University

Principal supervisor :

Mirco Ravanelli

drew.wagner@mila.quebec

nadhir.hassen@mila.quebec

Mojtaba Faramarzi

PhD - Université de Montréal

faramarm@mila.quebec

Parviz Haggi Mani

Independent visiting researcher - -

parviz.haggi@gmail.com

Nadhir Hassen

Collaborating researcher - Université de Montréal

Niklas Herbster

Research Intern - Université de Montréal

Co-supervisor :

Gauthier Gidel

niklas.herbster@gmail.com

prateek.humane@mila.quebec

Mai-Thi Ho

Master's Research - Université de Montréal

mai-thi.ho@mila.quebec

Prateek Humane

PhD

arnav-kumar.jain@mila.quebec

Nizar Islah

PhD - Université de Montréal

Principal supervisor :

Eilif B. Muller

nizar.islah@mila.quebec

PhD - Université de Montréal

PhD - Université de Montréal

ayush.kaushal@mila.quebec

Arian Khorasani

Collaborating researcher - Université de Montréal

arian.khorasani@mila.quebec

Neeraj Kumar

Collaborating Alumni - Université de Montréal

neeraj.kumar@mila.quebec

Gwen Legate

PhD - Concordia University

Principal supervisor :

gwendolyne.legate@mila.quebec

David Lemay

Master's Research - Université de Montréal

david.lemay@mila.quebec

amin.mansouri@mila.quebec

Amin Mansouri

Collaborating Alumni - Université de Montréal

Collaborating researcher

memariaa@mila.quebec

Andrei Mircea

PhD - Université de Montréal

Collaborating researcher - Université de Montréal

diganta.misra@mila.quebec

motahareh.pourrahimi@mila.quebec

Gabriela Moisescu-Pareja

Collaborating researcher - McGill University

Principal supervisor :

Doina Precup

moisescg@mila.quebec

Timothy Nest

PhD - Université de Montréal

Co-supervisor :

Eilif B. Muller

timothy.nest@mila.quebec

Jonathan Pilault

Collaborating researcher - Polytechnique Montréal

PhD - McGill University

Principal supervisor :

Pouya Bashivan

Mahta Ramezanian

Master's Research - Université de Montréal

Co-supervisor :

Guillaume Dumas

mahta.ramezanian@mila.quebec

Matthew Riemer

PhD - Université de Montréal

matthew.riemer@mila.quebec

Alexis Roger

PhD - McGill University

Principal supervisor :

Blake Richards

alexis.roger@mila.quebec

munish.sathish-kumar@mila.quebec

Munish Sathish Kumar

Collaborating researcher

Zibo Shang

PhD - Université de Montréal

zibo.shang@mila.quebec

Vaibhav Singh

PhD - Concordia University

Principal supervisor :

vaibhav.singh@mila.quebec

Gopeshh Subbaraj

PhD - Université de Montréal

gopeshh.subbaraj@mila.quebec

PhD - Université de Montréal

Co-supervisor :

benjamin.therien@mila.quebec

Collaborating researcher - Université de Montréal

tejas.vaidhya@mila.quebec

andrew.williams@mila.quebec

Sihui Wei

Undergraduate - McGill University

sihui.wei@mila.quebec

Andrew Williams

PhD - Université de Montréal

Co-supervisor :

Nicolas Chapados

daria.yasafova@mila.quebec

Daria Yasafova

Master's Research - Université de Montréal

He Zhu

PhD - McGill University

he.zhu@mila.quebec

Publications

GitChameleon 2.0: Evaluating AI Code Generation Against Python Library Version Incompatibilities

Victor May

Samira Ebrahimi Kahou

Massimo Caccia

The rapid evolution of software libraries poses a considerable hurdle for code generation, necessitating continuous adaptation to frequent v… (see more)ersion updates while preserving backward compatibility. While existing code evolution benchmarks provide valuable insights, they typically lack execution-based evaluation for generating code compliant with specific library versions. To address this, we introduce GitChameleon 2.0, a novel, meticulously curated dataset comprising 328 Python code completion problems, each conditioned on specific library versions and accompanied by executable unit tests. GitChameleon 2.0 rigorously evaluates the capacity of contemporary large language models (LLMs), LLM-powered agents, code assistants, and RAG systems to perform version-conditioned code generation that demonstrates functional accuracy through execution. Our extensive evaluations indicate that state-of-the-art systems encounter significant challenges with this task; enterprise models achieving baseline success rates in the 48-51% range, underscoring the intricacy of the problem. By offering an execution-based benchmark emphasizing the dynamic nature of code libraries, GitChameleon 2.0 enables a clearer understanding of this challenge and helps guide the development of more adaptable and dependable AI code generation methods. We make the dataset and evaluation code publicly available at https://github.com/mrcabbage972/GitChameleonBenchmark.

2026-12-30

OpenReview (published)

Turning TS2Vec into a Foundation Model for Time Series Classification

Vasilii Feofanov

Zhenghan Tai

Andrei Ermakov

TS2Vec is one of the strongest self-supervised methods for time series classification, introduced in 2022. It is designed to be trained anew… (see more) on every dataset: self-supervised pre-training followed by linear probing. Since then, the field has shifted towards foundation models that reuse one encoder across datasets. Yet TS2Vec offers a compact convolutional architecture that operates directly on temporal samples and accepts variable-length inputs without resampling. We ask whether these properties can be retained when the encoder is pre-trained only once, and show that they can: TS2Vec can essentially be turned into a foundation model. Three changes suffice: pre-train on a synthetic corpus, feed the encoder a raw amplitude view and an instance-normalized shape view of the input through two small stems, and distill one specialist encoder per view into a single encoder. The result, TS2Vec-FM, is a 1.6M-parameter dilated convolutional network that, kept frozen, reaches 83.3% on UCR-128 and 75.3% on UEA-27 under linear probing, within 0.3 pp of MantisV2, the current state of the art, and above TS2Vec trained separately on each target dataset. What we find most interesting is how different TS2Vec-FM is from other foundation models in design: near-parity is reached without attention, without a patch tokenizer, and without intermediate-layer extraction, using an "old-school" convolutional encoder and a two-layer MLP stem.

2026-10-19

arXiv (preprint)

WhatIf-Bench: Benchmarking Intervention-Aware Time Series Forecasting

Arjun Ashok

Mihir Parmar

Palash Goyal

Bhavana Dalvi Mishra

Chun-Liang Li

Alexandre Drouin

Jinsung Yoon

Tomas Pfister

Recent time series foundation models (TSFMs) can consume covariates in-context, extending zero-shot forecasting to problems driven by extern… (see more)al factors. This enables scenario analysis: forecasting how a target responds to events that set covariates to chosen future values. Whether they can do this reliably is an open question. We introduce WhatIf-Bench, a benchmark measuring how well forecasters use covariates to forecast a target's response to future events. WhatIf-Bench is semi-synthetic: time series come from mechanistic simulators of data centers, power grids, road traffic, and water networks, with controlled covariate-target relationships grounded in each system. Each task presents a history of events intervening on covariates, then asks to forecast a future event that either replays seen values (Replay) or sets them to unseen ones (Novel). Varying the number of covariates, historical events, and event length yields 37,665 tasks. Evaluating TSFMs, frontier LLMs, and simple regression baselines, we find the LLMs reach the highest skill (improvement over a naive baseline), ahead of the best TSFMs. Every model forecasts well on Replay futures (48 to 85% skill) but drops to -7 to 50% on Novel futures, with the regression baselines falling the most. Skill on Novel futures also falls as events set more covariates jointly. More historical events raise skill on Novel futures across the board, but large differences remain. WhatIf-Bench reveals generalization to Novel futures as the core challenge, and offers a testbed toward arbitrary scenario analysis.

2026-10-18

arXiv (preprint)

GCAM-Gym: A Benchmark for Agent-Based Climate Policy Search

Tianyu Zhang

Zhenghan Tai

Xinyu Wang

Yoshua Bengio

Comparing climate-policy search methods requires accounting for differences in policy control, available observations, feedback timing, and … (see more)simulator budgets. We introduce GCAM-Gym, a Gymnasium-compatible environment that supports full-policy evaluation and stepwise control through a common interface and exposes the components of the policy objective. We compare seven search and control systems in three settings: target attainment and cost, cost-temperature trade-offs, and one-step delayed feedback. Each system is evaluated with ten seeds per setting, for a total of 3,210 attempted GCAM evaluations. Four systems share a budget of 24 attempts per seed; among these, CMA-ES achieves the highest interquartile mean (IQM) of the best returns from completed policy evaluations in all three settings. We release the evaluation archive and analysis tools to reconstruct the reported comparisons without running GCAM.

2026-10-15

arXiv (preprint)

What Does Layer-Importance Reveal About Transformers and State-Space Models?

Transformers and state-space models (SSMs) are the two dominant families of sequence models, and a central open question is how far the anal… (see more)ytical knowledge built for transformers transfers to SSMs. We address this through the lens of layer importance which underpins compression, selective fine-tuning, and interpretability across both families. We decompose layer importance into two distinct notions. \emph{Necessity} captures how much the pretrained model depends on a layer's existing contribution, measured by the loss increase from bypassing it. \emph{Plasticity} captures where the model absorbs new information during fine-tuning, measured by the magnitude of task-specific weight updates. Our analysis reveals that the two families behave fundamentally differently: in every evaluated residual transformer up to

2026-09-29

WiML @ Neural Information Processing Systems (published)

Temporal Recurrence Favors Fewer Layers

Ivan Anokhin

Johan Obando-Ceron

Sebastian Risi

In streaming tasks, recurrent models can carry latent computation across time, allowing each update to build on representations produced ear… (see more)lier. This raises a basic question: once temporal recurrence provides sequential computation across steps, how much depth is still needed within each step? Prior work has shown that recurrence can make shallow models competitive. We instead study this question as a compute-allocation problem, varying within-step depth, expert width, and the number of parallel experts per layer across several compute budgets. For each budget, we compare the best observed recurrent and non-recurrent allocations and the performance they achieve under approximately matched per-step computation. Across Sokoban and autoregressive FineWeb language modeling, we find that temporal recurrence shifts the best observed compute allocation toward substantially fewer layers, with comparable or better performance.

2026-09-10

arXiv (preprint)

What Survives KV-Cache Compression for Reasoning? A Dual-Channel View of Structural and Associative Memory

Md Rifat Arefin

Long reasoning traces make Transformer key–value (KV) caches grow linearly with generation length. Existing compression methods adopt diff… (see more)erent memory topologies, but it is unclear how those choices affect the information that survives eviction. We observe a task–fidelity mismatch: dense summaries achieve low KV reconstruction error yet lose information required for structured reasoning. We explain this through a dual-channel model of compressed memory consisting of a temporally organized scaffold and sparse associative bindings, suggesting that structure and association should be stored separately. We instantiate this idea as TT–Delta, which combines a Tensor-Train structural state with a Delta-rule associative state. On a frozen Qwen2.5-0.5B task model, TT–Delta compresses an evicted prefix by 25.3× (46,680 persistent scalars) and improves exact-answer generation on sequential linear-system reasoning from 0.302±0.113 to 0.839±0.171. In contrast, multi-head Delta is strongest on associative recall and branching graph search. These results suggest that no single compressed-memory topology is universally optimal; the appropriate topology depends on the structure of the reasoning state.

2026-08-06

Efficient Reasoning @ Conference on Language Modeling (published)

Beyond Naïve Prompting: Strategies for Improved Context-aided Forecasting with LLMs

Arjun Ashok

Andrew R. Williams

Vincent Zhihao Zheng

Nicolas Chapados

Étienne Marcotte

Valentina Zantedeschi

Alexandre Drouin

Real-world forecasting requires models to integrate not only historical data but also relevant contextual information provided in textual fo… (see more)rm. While large language models (LLMs) show promise for context-aided forecasting, critical challenges remain: we lack diagnostic tools to understand failure modes, performance remains far below their potential, and high computational costs limit practical deployment. We introduce a unified framework of four strategies that address these limitations along three orthogonal dimensions: model diagnostics, accuracy, and efficiency. Through extensive evaluation across model families from small open-source models to frontier models including Gemini, GPT, and Claude, we uncover both fundamental insights and practical solutions. Our findings span three key dimensions: diagnostic strategies reveal the “Execution Gap” where models correctly explain how context affects forecasts but fail to apply this reasoning; accuracy-focused strategies achieve substantial performance improvements of 25-50%; and efficiency-oriented approaches show that adaptive routing between small and large models can approach large model accuracy on average while significantly reducing inference costs. These orthogonal strategies can be flexibly integrated based on deployment constraints, providing practitioners with a comprehensive toolkit for practical LLM-based context-aided forecasting. Code is made available at https://github.com/ashok-arjun/beyond-naive-prompting.

2026-07-27

Transactions on Machine Learning Research (accepted)

Relevant and Irrelevant: A Renormalization Group Analysis of Transformer Attention

Parviz Haggi-Mani

Using the language of Wilsonian renormalization group theory (RG), we treat the Transformer's attention mechanism as a perturbation of the t… (see more)rained MLP residual-stack fixed point and ask whether it constitutes a relevant, marginal, or irrelevant operator. We derive a fixed-point shift formula and obtain four testable predictions for the fixed-point geometry, effective rank profile, layer specificity, and perturbation decay spectrum. Testing these on synthetic Markov chain sequences with controlled correlation length, we find: (1) For large chains(long correlation), attention is strongly relevant: it closes a residual loss gap the MLP cannot bridge and drives a phase transition in representation space, with effective rank jumping above input dimensionality at layer 1 and stabilizing at a high-dimensional plateau. (2) For short chains(short correlation), attention is irrelevant: the Transformer converges to the same loss and fixed-point geometry as the MLP, though it contracts perturbations faster. (3) The transition is dominated by the first-layer head (L0H0), which accounts for more than 4 times the representational shift of any subsequent head, consistent with the prediction that the relevant operator acts before the MLP begins integrating out positional variation. (4) Perturbation decay experiments reveal a regime reversal: in the long correlation regime the Transformer selectively preserves slow Markov modes (5.4 times the dynamic range in decay length vs. 1.3 times for the MLP); in the short correlation regime it suppresses all modes faster than the MLP, with no spectral selectivity. Together, these results show that the relevance of attention is not a property of the architecture but of the spectral structure of the data-generating process, and that a first-order RG perturbation framework provides a predictive account of that difference.

2026-07-15

arXiv (preprint)

When Geometry Aligns: Dihedral Hidden-State Transformations in UNet, ViT, and DiT Architectures

Mojtaba Faramarzi

Alex Lamb

Diffusion architectures now encompass convolutional UNets as well as transformer-based designs such as Diffusion Transformers (DiTs), inspir… (see more)ed by Vision Transformers (ViTs), yet the effects of structured geometric perturbations within these architectures remain poorly understood. We study this question through a unified framework that applies reflection-based elements of the dihedral group to intermediate hidden states as controlled internal interventions, contrasting geometrically consistent and inconsistent variants. Using activation-level diagnostics, including Self-Consistency Shift (SCS), Activation Mass Scatter (AMS), and Drift, we analyze feature stability and geometric drift. We find that consistent transformations improve stability, while inconsistent ones induce predictable, architecture-specific failures. In the main Stable Diffusion 2.1 U-Net study, we evaluate seven intervention modes over three seeds and complement the internal diagnostics with image-level FID, KID, CLIP score, and LPIPS diversity. Taken together with supporting ViT and controlled DiT analyses, these results establish geometric consistency as a key principle for stable hidden-state interventions in spatially structured vision and diffusion models.

2026-07-02

arXiv (preprint)

Formal Epistemic Structure Leaves Geometric Traces in LLMs: Evidence from S5 Multi-Agent Logic and RoBERTa

David John Lemay

Emil Sayilov

We ask whether the formal structure of S5 multi-agent epistemic logic leaves recoverable geometric traces in a fine-tuned language model. Va… (see more)n Benthem's product topology for S5 predicts that the state space of

2026-06-26

LogiSymb @ International Joint Conference on Artificial Intelligence (poster)

Feature Geometry of Language Models Transfer Across Modalities to Time Series

Zhenghan Tai

Vasilii Feofanov

Language models transfer to time-series forecasting, but it is unclear whether this reflects reusable internal structure or rapid relearning… (see more) under a familiar architecture. We study this transfer directly by comparing pretrained and randomly initialized versions of the same model on a forecasting objective whose inputs have little semantic overlap with text but still require autoregressive sequential structure. Across Qwen3-0.6B finetuning experiments, language initialization gives coherent per-example gradients from the first update, while random initialization first passes through a low-alignment warmup phase. Effective-rank and hidden-state analyses show that finetuning selectively reshapes an existing representation geometry rather than constructing the simpler temporal geometry found by models trained from scratch. Cross-domain sparse features and causal ablations then expose candidate transferred primitives, including a Layer~1 head--MLP circuit whose ablation selectively increases loss on periodic forecasting and repetitive language passages. These results support an account of cross-modal transfer in which autoregressive pretraining creates temporal feature geometry that can be selected and specialized outside language.

2026-06-10

ICML.cc/2026/Workshop/Mech_Interp (poster)