Portrait of Isabeau Prémont-Schwarz

Isabeau Prémont-Schwarz

Associate Academic Member
isabeau.premont-schwarz@mila.quebec
Assistant Professor, Université Laval, Computer science and software engineering
Adjunct Professor, McGill University, Computer Science
Research Topics
AGI (Artificial General Intelligence)
Brain-inspired AI
Deep Learning
Reasoning
Reinforcement Learning
Representation Learning
Website
LinkedIn

Biography

Isabeau is interested neuro-inspired AI, agent learning (including reinforcement learning), and how to build useful abstract representations which are able to generalize out of distribution.

Current Students

Melisa Civelekoğlu
Master's Research - Université Laval
melisa.civelekoglu@mila.quebec
Élodie Coté-Gauthier
Master's Research - McGill University
Principal supervisor :
Doina Precup
elodie.cote-gauthier@mila.quebec
Luc Coupal
PhD - Université Laval
luc.coupal@mila.quebec
Website
Github
LinkedIn
Varun Dhanraj
PhD - Université Laval
vdhanraj@uwaterloo.ca
Website
Github
LinkedIn
Google Scholar
Samy Mammeri
Undergraduate - Université Laval
Principal supervisor :
Christian Gagné
samy.mammeri@mila.quebec
Google Scholar
maxime Ouimet
Master's Research - Université Laval
maxime.ouimet@mila.quebec

Publications

Grid Cells as Spectral Successor Features: A Reusable Basis for Continual Reinforcement Learning
Élodie Coté-Gauthier
Doina Precup
Isabeau Prémont-Schwarz
A reinforcement-learning agent moving between tasks in a fixed environment faces a mismatch: the reward changes, but the geometry does not. … (see more)Methods that learn a representation and a value function jointly relearn both at every task switch, which is slow and prone to forgetting. We take the identification of grid cells with eigenvectors of the successor representation as a design principle, and build the feature basis from the low-frequency eigenvectors of the symmetrised random-walk graph Laplacian, defined over state-action pairs rather than states so that the features encode what an action does. Because this basis diagonalises the transition operator, the successor-feature transform is exactly diagonal with closed-form entries fixed by the Laplacian eigenvalues and the discount. No temporal-difference learning is needed for it, and neither a successor-feature network nor a replay buffer is required: the only learned quantity is a linear reward map, re-estimated per task. On two goal locations in a two-room maze sharing an identical transition structure, the resulting agent holds near-ceiling return across every task switch, where a successor-feature baseline that learns its features collapses at each change. The cost is a slightly lower ceiling imposed by the linear parameterisation. The basis here is computed from the true transition graph, so this arm is an oracle upper bound rather than a method competing on equal information; learning the basis online is in progress.
2026-09-29
WiML @ Neural Information Processing Systems (published)
openreview.net
openreview.net
TPR-Attention for Combinatorial Generalization
Melisa Civelekoğlu
Isabeau Prémont-Schwarz
Systematic generalization remains a significant challenge in deep learning. In particular, combinatorial generalization - generalizing to ne… (see more)w configurations of known factors of variation - is effortless for humans but difficult for standard neural architectures that rely on statistical correlations rather than explicit structural representations. We introduce a new architectural component that embeds structured inductive bias into deep learning: an attention mechanism operating over tensor-product representations (TPRs). Through controlled experiments on compositional tasks, we show that this TPR-attention mechanism outperforms existing architectural components in combinatorial generalization. These results highlight the value of integrating explicit compositional structure into neural attention and point toward a promising path for models capable of systematic generalization.
2026-08-30
arXiv (preprint)
doi.org
arxiv.org
Addressing Sparse-Rewards in RL with Scalable Hierarchical Novel Eigen Options
Priyesh Vijayan
Élodie Coté-Gauthier
Mathieu Reymond
Sarath Chandar
Doina Precup
Isabeau Prémont-Schwarz
Temporally extended exploration via graph Laplacian-based options is a promising approach to sparse-reward reinforcement learning (RL), but … (see more)existing methods either do not explicitly target novelty or fail to scale to pixel-based domains under function approximation. Novel Exploration via Orthogonality (NEO) addresses the first issue by constructing options that navigate from highly visited regions toward less visited ones, yet prior results were limited to settings where exact eigenvectors can be computed. We present a scalable extension of NEO to pixel-based domains, built on three contributions. First, we use a novelty-weighted continuous Laplacian graph-drawing objective, which enables RL with continuous observations. Second, we embed the resulting eigen-potential options within a hierarchical reinforcement learning framework, enabling coherent temporally extended behavior. Third, we observe that learned eigen-potential rewards are directional but locally unreliable under online approximation; we therefore augment each option reward with a novelty bonus, a novel design idea that proves essential for stabilizing option learning while preserving novelty-directed exploration. Together, these contributions yield stronger and more persistent exploration, enabling longer option rollouts and better access to hard-to-reach novel states. Empirically, our method significantly outperforms both the prior scalable Laplacian-option baseline and a direct extension of NEO on sparse-reward benchmarks under a fixed budget. On Montezuma's Revenge, our best variant achieves approximately 1.8x higher return than both baselines. On Venture, both baselines yield returns near zero, whereas our method achieves a return of 1135. Across seven hard ProcGen games, our method achieves approximately 3.5x and 5.6x higher aggregate normalized return than the two baselines, respectively.
2026-07-20
RLBRew @ Reinforcement Learning Conference (published)
openreview.net
openreview.net