Portrait of Doina Precup

Doina Precup

Core Academic Member
precupdo@mila.quebec
Canada CIFAR AI Chair
Associate Professor, McGill University, School of Computer Science
Research Team Leader, Google DeepMind
Research Topics
Medical Machine Learning
Molecular Modeling
Probabilistic Models
Reasoning
Reinforcement Learning

Biography

Doina Precup combines teaching at McGill University with fundamental research on reinforcement learning, in particular AI applications in areas of significant social impact, such as health care. She is interested in machine decision-making in situations where uncertainty is high.

In addition to heading the Montreal office of Google DeepMind, Precup is a Senior Fellow of the Canadian Institute for Advanced Research and a Fellow of the Association for the Advancement of Artificial Intelligence.

Her areas of speciality are artificial intelligence, machine learning, reinforcement learning, reasoning and planning under uncertainty, and applications.

Current Students

Safa Alver
PhD - McGill University
alversaf@mila.quebec
Samin Yeasar Arnob
PhD - McGill University
Website
Google Scholar
Sumana Basu
PhD - McGill University
Co-supervisor :
Adriana Romero Soriano
Website
Github
Google Scholar
Veronica Chelu
PhD - McGill University
Github
Lynn Cherif
Master's Research - McGill University
Co-supervisor :
Khimya Khetarpal
Website
Google Scholar
Raymond Chua
PhD - McGill University
Co-supervisor :
Blake Richards
Website
Google Scholar
Wesley Chung
PhD - McGill University
Principal supervisor :
David Meger
Jonathan Colaço Carr
Master's Research - McGill University
Principal supervisor :
Prakash Panangaden
Élodie Coté-Gauthier
Collaborating researcher - McGill University
Franco Del Balso
Research Intern - Université de Montréal
Github
Jesse Farebrother
PhD - McGill University
Principal supervisor :
Marc Gendron-Bellemare
Website
Github
Google Scholar
Tugce Gurbuz
PhD - McGill University
Principal supervisor :
Eilif Benjamin Muller
Website
Github
Google Scholar
Howard Huang
PhD - McGill University
Haque Ishfaq
PhD - McGill University
Website
Github
Google Scholar
Mohammad Sami Nur Islam Islam
Master's Research - McGill University
Github
Google Scholar
Arushi Jain
PhD - McGill University
Website
Github
Google Scholar
Martin Klissarov
PhD - McGill University
Website
Github
Google Scholar
Flemming Kondrup
Postdoctorate - McGill University
Google Scholar
Elaine Lau
Master's Research - McGill University
Jonathan Lebensold
Collaborating Alumni - McGill University
Website
Github
Google Scholar
Jeremy Lucas
Undergraduate - McGill University
Ray Luo
PhD - McGill University
Principal supervisor :
Xujie Si
Website
Github
Google Scholar
G McCracken
PhD - McGill University
Google Scholar
Nazanin Mohammadi Sepahvand
PhD - McGill University
Github
Shahrad Mohammadzadeh
Master's Research - McGill University
Principal supervisor :
Reihaneh Rabbany
Website
Github
Google Scholar
Gabriela Moisescu-Pareja
Master's Research - McGill University
Padideh Nouri
PhD - Université de Montréal
Co-supervisor :
Yoshua Bengio
Website
Github
Google Scholar
Charles Onu
PhD - McGill University
Website
Github
Google Scholar
Gandharv Patil
PhD - McGill University
Co-supervisor :
Pablo Samuel Castro
Website
Github
Google Scholar
Nate Rahn
PhD - McGill University
Principal supervisor :
Marc Gendron-Bellemare
Sahand Rezaei-Shoshtari
PhD - McGill University
Co-supervisor :
David Meger
Website
Github
Google Scholar
Manoosh Samiei
PhD - McGill University
Co-supervisor :
Paul Masset
Website
Github
Google Scholar
Mandana Samiei
PhD - McGill University
Co-supervisor :
Blake Richards
samiemandana@gmail.com
Website
Github
Google Scholar
Shishir Sharma
PhD - McGill University
Website
Github
Google Scholar
Nishanth Anand Vemgal
PhD - McGill University
Website
Github
Google Scholar
David Anthony Venuto
PhD - McGill University
Priyesh Vijayan
PhD - McGill University
Co-supervisor :
Samira Ebrahimi Kahou
Website
Github
Google Scholar
Guangyuan Wang
Research Intern - McGill University
Github
Steve Wen
Master's Research - McGill University
Co-supervisor :
Gregory Dudek
Github
Zijing Wu
PhD - McGill University
Co-supervisor :
Paul Masset
Website
Github
Google Scholar
Shuyuan Zhang
PhD - McGill University
Github
Harry Zhao
PhD - McGill University
Co-supervisor :
Yoshua Bengio
Website
Github
Google Scholar

Publications

Understanding Behavioral Metric Learning: A Large-Scale Study on Distracting Reinforcement Learning Environments
Ziyan Luo
Tianwei Ni
Pierre-Luc Bacon
Doina Precup
Xujie Si
2025-05-31
ArXiv (preprint)
arxiv.org
arxiv.org
SCAR: Shapley Credit Assignment for More Efficient RLHF
Meng Cao
Shuyuan Zhang
Xiaojun Chang
Doina Precup
2025-05-26
ArXiv (preprint)
arxiv.org
arxiv.org
Uncovering a Universal Abstract Algorithm for Modular Addition in Neural Networks
Gavin McCracken
Gabriela Moisescu-Pareja
Vincent Létourneau
Doina Precup
Jonathan Love
We propose a testable universality hypothesis, asserting that seemingly disparate neural network solutions observed in the simple task of mo… (see more)dular addition are unified under a common abstract algorithm. While prior work interpreted variations in neuron-level representations as evidence for distinct algorithms, we demonstrate - through multi-level analyses spanning neurons, neuron clusters, and entire networks - that multilayer perceptrons and transformers universally implement the abstract algorithm we call the approximate Chinese Remainder Theorem. Crucially, we introduce approximate cosets and show that neurons activate exclusively on them. Furthermore, our theory works for deep neural networks (DNNs). It predicts that universally learned solutions in DNNs with trainable embeddings or more than one hidden layer require only O(log n) features, a result we empirically confirm. This work thus provides the first theory-backed interpretation of multilayer networks solving modular addition. It advances generalizable interpretability and opens a testable universality hypothesis for group multiplication beyond modular addition.
2025-05-23
ArXiv (preprint)
arxiv.org
arxiv.org
Uncovering a Universal Abstract Algorithm for Modular Addition in Neural Networks
Gavin McCracken
Gabriela Moisescu-Pareja
Vincent Létourneau
Doina Precup
Jonathan Love
We propose a testable universality hypothesis, asserting that seemingly disparate neural network solutions observed in the simple task of mo… (see more)dular addition are unified under a common abstract algorithm. While prior work interpreted variations in neuron-level representations as evidence for distinct algorithms, we demonstrate - through multi-level analyses spanning neurons, neuron clusters, and entire networks - that multilayer perceptrons and transformers universally implement the abstract algorithm we call the approximate Chinese Remainder Theorem. Crucially, we introduce approximate cosets and show that neurons activate exclusively on them. Furthermore, our theory works for deep neural networks (DNNs). It predicts that universally learned solutions in DNNs with trainable embeddings or more than one hidden layer require only O(log n) features, a result we empirically confirm. This work thus provides the first theory-backed interpretation of multilayer networks solving modular addition. It advances generalizable interpretability and opens a testable universality hypothesis for group multiplication beyond modular addition.
2025-05-23
ArXiv (preprint)
arxiv.org
arxiv.org
Plasticity as the Mirror of Empowerment
David Abel
Michael Bowling
Andre Barreto
Will Dabney
Shi Dong
Steven Hansen
Anna Harutyunyan
Khimya Khetarpal
Clare Lyle
Razvan Pascanu
Georgios Piliouras
Doina Precup
Jonathan Richens
Mark Rowland
Tom Schaul
Satinder Singh
2025-05-15
ArXiv (preprint)
arxiv.org
arxiv.org
Plasticity as the Mirror of Empowerment
David Abel
Michael Bowling
Andre Barreto
Will Dabney
Shi Dong
Steven Hansen
Anna Harutyunyan
Khimya Khetarpal
Clare Lyle
Razvan Pascanu
Georgios Piliouras
Doina Precup
Jonathan Richens
Mark Rowland
Tom Schaul
Satinder Singh
2025-05-15
ArXiv (preprint)
arxiv.org
arxiv.org
Language Agents Mirror Human Causal Reasoning Biases. How Can We Help Them Think Like Scientists?
Anthony GX-Chen
Dongyan Lin
Mandana Samiei
Doina Precup
Blake Richards
Rob Fergus
Kenneth Marino
2025-05-14
ArXiv (preprint)
arxiv.org
arxiv.org
Language Agents Mirror Human Causal Reasoning Biases. How Can We Help Them Think Like Scientists?
Anthony GX-Chen
Dongyan Lin
Mandana Samiei
Doina Precup
Blake Richards
Rob Fergus
Kenneth Marino
Language model (LM) agents are increasingly used as autonomous decision-makers who need to actively gather information to guide their decisi… (see more)ons. A crucial cognitive skill for such agents is the efficient exploration and understanding of the causal structure of the world -- key to robust, scientifically grounded reasoning. Yet, it remains unclear whether LMs possess this capability or exhibit systematic biases leading to erroneous conclusions. In this work, we examine LMs' ability to explore and infer causal relationships, using the well-established"Blicket Test"paradigm from developmental psychology. We find that LMs reliably infer the common, intuitive disjunctive causal relationships but systematically struggle with the unusual, yet equally (or sometimes even more) evidenced conjunctive ones. This"disjunctive bias"persists across model families, sizes, and prompting strategies, and performance further declines as task complexity increases. Interestingly, an analogous bias appears in human adults, suggesting that LMs may have inherited deep-seated reasoning heuristics from their training data. To this end, we quantify similarities between LMs and humans, finding that LMs exhibit adult-like inference profiles (but not children-like). Finally, we propose a test-time sampling method which explicitly samples and eliminates hypotheses about causal relationships from the LM. This scalable approach significantly reduces the disjunctive bias and moves LMs closer to the goal of scientific, causally rigorous reasoning.
2025-05-14
ArXiv (preprint)
arxiv.org
arxiv.org
Understanding the Effectiveness of Learning Behavioral Metrics in Deep Reinforcement Learning
Ziyan Luo
Tianwei Ni
Pierre-Luc Bacon
Doina Precup
Xujie Si
A key approach to state abstraction is approximating behavioral metrics (notably, bisimulation metrics) in the observation space, and embed … (see more)these learned distances in the representation space. While promising for robustness to task-irrelevant noise shown in prior work, accurately estimating these metrics remains challenging, requiring various design choices that create gaps between theory and practice. Prior evaluations focus mainly on final returns, leaving the quality of learned metrics and the source of performance gains unclear. To systematically assess how metric learning works in deep RL, we evaluate five recent approaches. We unify them under isometric embedding, identify key design choices, and benchmark them with baselines across 20 state-based and 14 pixel-based tasks, spanning 250+ configurations with diverse noise settings. Beyond final returns, we introduce the denoising factor to quantify the encoder’s ability to filter distractions. To further isolate the effect of metric learning, we propose an isolated metric estimation setting, where the encoder is influenced solely by the metric loss. Our results show that metric learning improves return and denoising only marginally, as its benefits fade when key design choices, such as layer normalization and self-prediction loss, are incorporated into the baseline. We also find that commonly used benchmarks (e.g., grayscale videos, varying state-based Gaussian noise dimensions) add little difficulty, while Gaussian noise with random projection and pixel-based Gaussian noise remain challenging even for the best methods. Finally, we release an open-source, modular codebase to improve reproducibility and support future research on metric learning in deep RL.
2025-05-09
rl-conference.cc/RLC/2025/Conference (accepted)
openreview.net
openreview.net
Understanding the Effectiveness of Learning Behavioral Metrics in Deep Reinforcement Learning
Ziyan Luo
Tianwei Ni
Pierre-Luc Bacon
Doina Precup
Xujie Si
A key approach to state abstraction is approximating behavioral metrics (notably, bisimulation metrics) in the observation space, and embed … (see more)these learned distances in the representation space. While promising for robustness to task-irrelevant noise shown in prior work, accurately estimating these metrics remains challenging, requiring various design choices that create gaps between theory and practice. Prior evaluations focus mainly on final returns, leaving the quality of learned metrics and the source of performance gains unclear. To systematically assess how metric learning works in deep RL, we evaluate five recent approaches. We unify them under isometric embedding, identify key design choices, and benchmark them with baselines across 20 state-based and 14 pixel-based tasks, spanning 250+ configurations with diverse noise settings. Beyond final returns, we introduce the denoising factor to quantify the encoder’s ability to filter distractions. To further isolate the effect of metric learning, we propose an isolated metric estimation setting, where the encoder is influenced solely by the metric loss. Our results show that metric learning improves return and denoising only marginally, as its benefits fade when key design choices, such as layer normalization and self-prediction loss, are incorporated into the baseline. We also find that commonly used benchmarks (e.g., grayscale videos, varying state-based Gaussian noise dimensions) add little difficulty, while Gaussian noise with random projection and pixel-based Gaussian noise remain challenging even for the best methods. Finally, we release an open-source, modular codebase to improve reproducibility and support future research on metric learning in deep RL.
2025-05-09
rl-conference.cc/RLC/2025/Conference (published)
openreview.net
openreview.net
Generative AI: Hype, Hope, and Responsible Use in Science and Everyday Life
Doina Precup
2025-05-01
Biological Psychiatry (published)
doi.org
Rejecting Hallucinated State Targets during Planning
Harry Zhao
Mingde Zhao
Tristan Sylvain
Romain Laroche
Doina Precup
Yoshua Bengio
2025-05-01
ICML.cc/2025/Conference (poster)
openreview.net
openreview.net