Portrait of Guillaume Lajoie

Guillaume Lajoie

Core Academic Member

guillaume.lajoie@mila.quebec

Canada CIFAR AI Chair

Associate Professor, Université de Montréal, Department of Mathematics and Statistics

Visiting Researcher, Google

Research Topics

AI for Science

AI in Health

Cognition

Computational Neuroscience

Deep Learning

Dynamical Systems

Optimization

Reasoning

Recurrent Neural Networks

Representation Learning

Biography

Guillaume Lajoie is an Associate professor in the Department of Mathematics and Statistics at Université de Montréal and a Core Academic Member of Mila – Quebec Artificial Intelligence Institute. He holds a Canada-CIFAR AI Research Chair, and a Canada Research Chair (CRC) in Neural Computation and Interfacing.

His research is positioned at the intersection of AI and Neuroscience where he develops tools to better understand mechanisms of intelligence common to both biological and artificial systems. His research group's contributions range from advances in multi-scale learning paradigms for large artificial systems, to applications in neurotechnology. Dr. Lajoie is actively involved in responsible AI development efforts, seeking to identify guidelines and best practices for use of AI in research and beyond.

Current Students

Federico Arangath Joseph

Collaborating researcher - ETH Zurich

Collaborating Alumni - Polytechnique Montréal

Independent visiting researcher

Principal supervisor :

Sangnie Bhardwaj

PhD - Université de Montréal

Co-supervisor :

Hugo Larochelle

Colin Bredenberg

Postdoctorate - Université de Montréal

Co-supervisor :

PhD - Université de Montréal

Postdoctorate - Université de Montréal

Co-supervisor :

PhD - Université de Montréal

Principal supervisor :

PhD - Université de Montréal

Postdoctorate - McGill University

Principal supervisor :

Master's Research - Polytechnique Montréal

Principal supervisor :

Marco Bonizzato

Nanda Harishankar Krishna

PhD - Université de Montréal

Independent visiting researcher - McGill University

PhD - McGill University

Principal supervisor :

Thomas Jiralerspong

PhD - Université de Montréal

Co-supervisor :

Master's Research - Université de Montréal

Co-supervisor :

PhD - McGill University

Principal supervisor :

Mathys Loiselle

Research Intern - Concordia University

Co-supervisor :

PhD - Université de Montréal

Co-supervisor :

Abdel Mfougouon Njupoun

PhD - Université de Montréal

Co-supervisor :

PhD - Université de Montréal

Co-supervisor :

Alexandre Payeur

Collaborating researcher - Université de Montréal

Mohammad Pezeshki

Collaborating researcher

Principal supervisor :

Master's Research - Université de Montréal

Mauricio Rivera

Master's Research - Université de Montréal

Principal supervisor :

Marco Bonizzato

PhD - Université de Montréal

Principal supervisor :

Pravish Sainath

PhD - Université de Montréal

Co-supervisor :

Postdoctorate - Université de Montréal

Ezekiel Williams

PhD - Université de Montréal

Independent visiting researcher - University of South California

Blog Posts

Représentation graphique d'un nerf vague

May 21, 2025

Machine Learning for the Segmentation of Different Nerve Fibre Activations from Brain-to-body Neural Signals

by

Olivier Tessier-Larivière

Pascal Fortier-Poisson

Guillaume Lajoie

Read the article

Visuel abstrait

June 13, 2024

What Do Synaptic Weight Distributions Tell Us About Learning in the Brain ?

by

Jonathan Cornford

Arna Ghosh

Guillaume Lajoie

Read the article

Publications

Learning Long-term Dependencies Using Cognitive Inductive Biases in Self-attention RNNs

Bhargav Kanuparthi

Guillaume Lajoie

Attention and self-attention mechanisms, inspired by cognitive processes, are now central to state-of-the-art deep learning on sequential ta… (see more)sks. However, most recent progress hinges on heuristic approaches that rely on considerable memory and computational resources that scale poorly. In this work, we propose a relevancy screening mechanism, inspired by the cognitive process of memory consolidation, that allows for a scalable use of sparse self-attention with recurrence. We use simple numerical experiments to demonstrate that this mechanism helps enable recurrent systems on generalization and transfer learning tasks. Based on our results, we propose a concrete direction of research to improve scalability and generalization of attentive recurrent networks.

2019-12-31

(published)

www.semanticscholar.org

Non-normal Recurrent Neural Network (nnRNN): learning long time dependencies while improving expressivity with transient dynamics

Maximilian Puelma Touzel

Eugene Vorontsov

Guillaume Lajoie

A recent strategy to circumvent the exploding and vanishing gradient problem in RNNs, and to allow the stable propagation of signals over lo… (see more)ng time scales, is to constrain recurrent connectivity matrices to be orthogonal or unitary. This ensures eigenvalues with unit norm and thus stable dynamics and training. However this comes at the cost of reduced expressivity due to the limited variety of orthogonal transformations. We propose a novel connectivity structure based on the Schur decomposition and a splitting of the Schur form into normal and non-normal parts. This allows to parametrize matrices with unit-norm eigenspectra without orthogonality constraints on eigenbases. The resulting architecture ensures access to a larger space of spectrally constrained matrices, of which orthogonal matrices are a subset. This crucial difference retains the stability advantages and training speed of orthogonal RNNs while enhancing expressivity, especially on tasks that require computations over ongoing input sequences.

2019-09-05

NeurIPS.cc/2019/Reproducibility_Challenge (unknown)