Irina Rish

Biography

Irina Rish is a full professor at the Université de Montréal (UdeM), where she leads the Autonomous AI Lab, and a core academic member of Mila – Quebec Artificial Intelligence Institute.

In addition to holding a Canada Excellence Research Chair (CERC) and a CIFAR Chair, she leads the U.S. Department of Energy’s INCITE project on Scalable Foundation Models on Summit & Frontier supercomputers at the Oak Ridge Leadership Computing Facility. She co-founded and serves as CSO of Nolano.ai.

Rish’s current research interests include neural scaling laws and emergent behaviors (capabilities and alignment) in foundation models, as well as continual learning, out-of-distribution generalization and robustness.

Before joining UdeM in 2019, she was a research scientist at the IBM T.J. Watson Research Center, where she worked on various projects at the intersection of neuroscience and AI, and led the Neuro-AI challenge. She was awarded the IBM Eminence & Excellence Award and IBM Outstanding Innovation Award (2018), IBM Outstanding Technical Achievement Award (2017) and IBM Research Accomplishment Award (2009).

She holds 64 patents and has published 120 research papers, several book chapters, three edited books and a monograph on sparse modeling.

Current Students

Ivan Anokhin

PhD - Université de Montréal

Co-supervisor :

Samira Ebrahimi Kahou

ivan.anokhin@mila.quebec

rishika.bhagwatkar@mila.quebec

Rifat Arefin

PhD - Université de Montréal

Arjun Ashok

PhD - Université de Montréal

Co-supervisor :

Alexandre Drouin

arjun.ashok@mila.quebec

Master's Research - Université de Montréal

PhD - Université de Montréal

ethan.caballero@mila.quebec

Pietro Cagnasso

PhD - Concordia University

Principal supervisor :

pietro.cagnasso@mila.quebec

Mohammad Javad Darvishi Bayazi

Amin Darabi

PhD - Université de Montréal

amin.darabi@mila.quebec

Collaborating researcher - Université de Montréal

mj.darvishi92@gmail.com

Wagner Drew

Master's Research - Concordia University

Principal supervisor :

Mirco Ravanelli

drew.wagner@mila.quebec

kshitij.gupta@mila.quebec

Mojtaba Faramarzi

PhD - Université de Montréal

faramarm@mila.quebec

Kshitij Gupta

Collaborating researcher

Co-supervisor :

Sarath Chandar

Parviz Haggi Mani

Independent visiting researcher - -

parviz.haggi@gmail.com

Nadhir Hassen

Collaborating researcher - Université de Montréal

nadhir.hassen@mila.quebec

Niklas Herbster

Research Intern - Université de Montréal

Co-supervisor :

Gauthier Gidel

niklas.herbster@gmail.com

prateek.humane@mila.quebec

Prateek Humane

PhD

arnav-kumar.jain@mila.quebec

Nizar Islah

PhD - Université de Montréal

Principal supervisor :

Eilif B. Muller

nizar.islah@mila.quebec

PhD - Université de Montréal

PhD - Université de Montréal

ayush.kaushal@mila.quebec

Arian Khorasani

Collaborating researcher - Université de Montréal

arian.khorasani@mila.quebec

Neeraj Kumar

Collaborating Alumni - Université de Montréal

neeraj.kumar@mila.quebec

Gwen Legate

PhD - Concordia University

Principal supervisor :

gwendolyne.legate@mila.quebec

David Lemay

Master's Research - Université de Montréal

david.lemay@mila.quebec

amin.mansouri@mila.quebec

Amin Mansouri

Collaborating Alumni - Université de Montréal

Collaborating researcher

memariaa@mila.quebec

Andrei Mircea

PhD - Université de Montréal

Collaborating researcher - Université de Montréal

diganta.misra@mila.quebec

motahareh.pourrahimi@mila.quebec

Gabriela Moisescu-Pareja

Collaborating researcher - McGill University

Principal supervisor :

Doina Precup

moisescg@mila.quebec

Timothy Nest

PhD - Université de Montréal

Co-supervisor :

Eilif B. Muller

timothy.nest@mila.quebec

Jonathan Pilault

Collaborating researcher - Polytechnique Montréal

PhD - McGill University

Principal supervisor :

Pouya Bashivan

Mahta Ramezanian

Master's Research - Université de Montréal

Co-supervisor :

Guillaume Dumas

mahta.ramezanian@mila.quebec

Matthew Riemer

PhD - Université de Montréal

matthew.riemer@mila.quebec

Alexis Roger

PhD - McGill University

Principal supervisor :

Blake Richards

alexis.roger@mila.quebec

munish.sathish-kumar@mila.quebec

Munish Sathish Kumar

Collaborating researcher

Zibo Shang

PhD - Université de Montréal

zibo.shang@mila.quebec

Vaibhav Singh

PhD - Concordia University

Principal supervisor :

vaibhav.singh@mila.quebec

Gopeshh Subbaraj

PhD - Université de Montréal

gopeshh.subbaraj@mila.quebec

PhD - Université de Montréal

Co-supervisor :

benjamin.therien@mila.quebec

Collaborating researcher - Université de Montréal

tejas.vaidhya@mila.quebec

andrew.williams@mila.quebec

Sihui Wei

Undergraduate - McGill University

sihui.wei@mila.quebec

Andrew Williams

PhD - Université de Montréal

Co-supervisor :

Nicolas Chapados

daria.yasafova@mila.quebec

Daria Yasafova

Master's Research - Université de Montréal

He Zhu

PhD - McGill University

he.zhu@mila.quebec

Publications

Unified Neural Scaling Laws

Ethan Caballero

Priyank Jaini

David Krueger

We present a functional form (that we refer to as a Unified Neural Scaling Law (UNSL)) that accurately models and extrapolates the scaling b… (see more)ehaviors of deep neural networks as multiple dimensions all vary simultaneously (i.e. how the evaluation metric of interest varies as one simultaneously varies the number of model parameters, training dataset size, number of training steps, number of inference steps, amount of compute, and various hyperparameters) for various architectures and for each of various tasks within a varied set of upstream and downstream tasks. This set includes large-scale vision, language, math, and reinforcement learning. When compared to other functional forms for neural scaling, this functional form yields extrapolations of scaling behavior that are considerably more accurate on this set.

2026-05-24

arXiv (preprint)

arxiv.org

LLM Pretraining Shapes a Generalizable Manifold: Insights into Cross-Modal Transfer to Time Series

Zhenghan Tai

Vasilii Feofanov

Can language-pretrained transformers become effective time-series forecasters, and why? In this paper, we show that cross-modal transfer ari… (see more)ses because language pretraining preconditions time series training with a reusable manifold. A linear probe on frozen LLM states decodes realistic time-series trajectories without paired supervision, and retrieval in this projected space yields competitive forecasts, showing that structure and dynamics exist before finetuning. Pretrained initialization also improves optimization, producing coherent gradients and a highly anisotropic loss landscape unlike random initialization. Finetuning then acts as low-dimensional alignment, reusing existing directions rather than learning temporal primitives from scratch, as evidenced by low-rank updates, subspace alignment, and shared features for periodicity, trend, and repetition. Together, these results support a geometric account of LLM-to-time-series transfer: language pretraining builds the manifold, and finetuning projects numerical dynamics onto task-relevant directions.

2026-05-18

arXiv (preprint)

arxiv.org

World models, artificial general intelligence and the hard problems of life–mind continuity: toward a unified understanding of natural and artificial intelligence

Adam Safron

Michael Levin

Victoria Klimaj

Zahra Sheikhbahaee

Dalton Sakthivadivel

Adeel Razi

David Ha

Nick Hay

Kevin Schmidt

David Krakauer

Melanie Mitchell

Samuel J. Gershman

Joshua B. Tenenbaum

Abstract This special issue examines how natural and artificial intelligences (AIs) model the world, and what this modelling reveals about c… (see more)ognition and relationships between life and mind. Rather than adopting a single definition, the collection considers how world models function and emerge in biological and artificial systems, exploring a diverse range of world modelling including causal, self-referential, individual goal-directed, collective and narrative forms. A recurring theme is the extent to which current AI systems trained on vast quantities of data learn the context-sensitive, temporally embedded, value-laden dimensions of world modelling that characterize diverse biological intelligences, or whether their impressive capabilities arise primarily from statistical surface regularities. The contributions also raise broader issues concerning embodiment, complexity, learning architectures and the social and scientific contexts in which world models operate. With this collection, we hope to clarify the conceptual landscape, identify key points of similarity and divergence between natural and artificial minds, and outline questions that may guide future research on the forms of world modelling that support grounded understanding, robust agency and potentially human-like general intelligence. This article is part of the theme issue ‘World models in natural and artificial intelligence’.

2026-05-13

Philosophical Transactions of Royal Society A: Mathematical, Physical and Engineering Sciences (published)

Adaptive Batch Sizes Using Non-Euclidean Gradient Noise Scales for Stochastic Sign and Spectral Descent

Hiroki Naganuma

Shagun Gupta

Youssef Briki

Ioannis Mitliagkas

Parameswaran Raman

Hao-Jun Michael Shi

To maximize hardware utilization, modern machine learning systems typically employ large constant or manually tuned batch size schedules, re… (see more)lying on heuristics that are brittle and costly to tune. Existing adaptive strategies based on gradient noise scale (GNS) offer a principled alternative. However, their assumption of SGD's Euclidean geometry creates a fundamental mismatch with popular optimizers based on generalized norms, such as signSGD / Signum (

2026-04-29

International Conference on Machine Learning (accepted)

Warming Up for Zeroth-Order Federated Pre-Training with Low Resource Clients

Gwen Legate

Federated learning enables collaborative model training across numerous edge devices without requiring participants to share data; however, … (see more)memory and communication constraints on these edge devices may preclude their participation in training. We consider a setting in which a subset of edge devices are below a critical memory or communication threshold required to conduct model updates. Under typical federated optimization algorithms, these devices are excluded from training which renders their data inaccessible and increases system induced bias. We are inspired by MeZO, a zeroth-order method used for memory-efficient fine-tuning. The increased variance inherent to zeroth-order gradient approximations has relegated previous zeroth-order optimizers exclusively to the domain of fine tuning; a limitation we seek to correct. We devise a federated, memory-efficient zeroth-order optimizer, ZOWarmUp that permits zeroth-order training from a random initialization. ZOWarmUp leverages differing client capabilities and careful variance reduction techniques to facilitate participation of under-represented, low-resource clients in model training. Like other federated zeroth-order methods, ZOWarmUp eliminates the need for edge devices to transmit their full gradients to the server and instead relies on only a small set of random seeds, rendering the up-link communication cost negligible. We present experiments using various datasets and model architectures to show that ZOWarmUp is a robust algorithm that can can be applied under a wide variety of circumstances. For systems with a high proportion of edge devices that would otherwise be excluded from training, this algorithm provides access to a greater volume and diversity of data, thus improving training outcomes.

2026-04-05

Conference on Lifelong Learning Agents (published)

proceedings.mlr.press

Emergent Reasoning via Recursive Latent Reinforcement Pretraining

Large language models (LLMs) often rely on explicit chain-of-thought (CoT) traces to solve multi-step reasoning problems, but these traces i… (see more)ncrease inference cost, expose brittle prompt dependence, and complicate training objectives. We study an alternative: \emph{latent deliberation} implemented as a small recurrent refinement module that performs multiple internal ``thinking`` steps while keeping the external sequence length fixed. We introduce \textbf{Recursive Latent Reinforcement Pretraining (RLRP)}, a training recipe that augments a base causal LLM with a shared latent head executed for

2026-03-04

LLM_Reasoning @ International Conference on Learning Representations (published)

AIF-GEN: Open-Source Platform and Synthetic Dataset Suite for Reinforcement Learning on Large Language Models

Jacob Chmura

Shahrad Mohammadzadeh

Taz Scott-Talib

Nishanth Anand

2026-03-01

LLA @ International Conference on Learning Representations (poster)

On the Adversarial Robustness of Discrete Image Tokenizers

Rishika Bhagwatkar

Nicolas Flammarion

Francesco Croce

Discrete image tokenizers encode visual inputs as sequences of tokens from a finite vocabulary and are gaining popularity in multimodal syst… (see more)ems, including encoder-only, encoder-decoder, and decoder-only models. However, unlike CLIP encoders, their vulnerability to adversarial attacks has not been explored. Ours being the first work studying this topic, we first formulate attacks that aim to perturb the features extracted by discrete tokenizers, and thus change the extracted tokens. These attacks are computationally efficient, application-agnostic, and effective across classification, multimodal retrieval, and captioning tasks. Second, to defend against this vulnerability, inspired by recent work on robust CLIP encoders, we fine-tune popular tokenizers with unsupervised adversarial training, keeping all other components frozen. While unsupervised and task-agnostic, our approach significantly improves robustness to both unsupervised and end-to-end supervised attacks and generalizes well to unseen tasks and data. Unlike supervised adversarial training, our approach can leverage unlabeled images, making it more versatile. Overall, our work highlights the critical role of tokenizer robustness in downstream tasks and presents an important step in the development of safe multimodal foundation models.

2026-02-19

arXiv (preprint)

arxiv.org

$\mu$LO: Compute-Efficient Meta-Generalization of Learned Optimizers

Benjamin Therien

Charles-Etienne Joseph

Learned optimizers (LOs) have the potential to significantly reduce the wall-clock training time of neural networks. However, they can strug… (see more)gle to optimize unseen tasks (*meta-generalize*), especially when training networks wider than those seen during meta-training. To address this, we derive the Maximal Update Parametrization (

2025-12-31

International Conference on Learning Representations (Accept (Poster))

Deep neural networks divide and conquer dihedral multiplication

Sihui Wei

Gavin McCracken

Gabriela Moisescu-Pareja

Harley Wiltzer

Doina Precup

Jonathan Love

We find multilayer perceptrons and transformers both universally learn an instantiation of the same divide-and-conquer algorithm that requir… (see more)es only a logarithmic number of neural representations to solve dihedral multiplication. Clustering neurons based on similar activation behaviour reveals remarkably clear structure: each neural representation corresponds to a Cayley graph. To our knowledge, this is the first work that fully characterizes and describes all neural representations that are learnable on a dataset, while prior work on group multiplications studied neuron-level behavior, or preliminarily investigated cluster behavior. Thus, we can understand the algorithm networks universally learn at three levels of abstraction: 1) Neurons activate on coset or approximate coset structure of the dihedral group. 2) Groups of neurons together form neural representations that act to divide the dataset into different subproblems, being Cayley graphs, where the equivalence class of the answer is computed. 3) The global algorithm then linearly combines each neural representation (subproblem) together at the logits. This work provides a deep case study and provides the community with a very well understood toy model for interpretability, as well as makes steps toward proving the conjecture that DNNs will divide and conquer all group multiplication tasks.

2025-12-31

International Conference on Machine Learning (Accept (regular))

GitChameleon 2.0: Evaluating AI Code Generation Against Python Library Version Incompatibilities

Victor May

Samira Ebrahimi Kahou

Massimo Caccia

The rapid evolution of software libraries poses a considerable hurdle for code generation, necessitating continuous adaptation to frequent v… (see more)ersion updates while preserving backward compatibility. While existing code evolution benchmarks provide valuable insights, they typically lack execution-based evaluation for generating code compliant with specific library versions. To address this, we introduce GitChameleon 2.0, a novel, meticulously curated dataset comprising 328 Python code completion problems, each conditioned on specific library versions and accompanied by executable unit tests. GitChameleon 2.0 rigorously evaluates the capacity of contemporary large language models (LLMs), LLM-powered agents, code assistants, and RAG systems to perform version-conditioned code generation that demonstrates functional accuracy through execution. Our extensive evaluations indicate that state-of-the-art systems encounter significant challenges with this task; enterprise models achieving baseline success rates in the 48-51% range, underscoring the intricacy of the problem. By offering an execution-based benchmark emphasizing the dynamic nature of code libraries, GitChameleon 2.0 enables a clearer understanding of this challenge and helps guide the development of more adaptable and dependable AI code generation methods. We make the dataset and evaluation code publicly available at https://github.com/mrcabbage972/GitChameleonBenchmark.

2025-12-31

Association for Computational Linguistics (published)

Position: Collusion Risks Among AI Reasoning Agents Justify Certification Requirements for Making Market Decisions

Matthew Riemer

Tommaso Tosato

Maximilian Puelma Touzel

This position paper argues that AI agents with chain-of-thought reasoning capabilities are predisposed to exhibit collusive behavior and sho… (see more)uld be required to obtain behavioral certification before making decisions that affect economic markets. This is because integrating these agents into society could collapse the legal evidentiary distinction between competition and collusion among independent firms without eroding the economic harm distinction. Experiments with DeepSeek-R1 agents in the Bertrand oligopoly pricing domain reveal a tendency towards tacit collusion that persists even when humans prompt the agents not to collude. We further show that the chain-of- thought of these agents can be steered toward either extremely collusive or highly competitive behavior in a way that is not semantically detectable by another LLM analyzing the reasoning traces. As a result, deploying reasoning agents for market decisions leads to collusive economic outcomes without any evidence of conspiracy or intent. Thus, certification based on observed behavior in representative situations is necessary to prevent collusion. We provide preliminary evidence that such agents can be steered in a generalizable way toward efficient competitive equilibria. However, developing a comprehensive behavioral certification will be required before these models can be deployed in real-world markets while ensuring their stability and efficiency.

2025-12-31

International Conference on Machine Learning (Accept (regular))