Guillaume Lajoie

matteo.alleman@mila.quebec

Biographie

Guillaume Lajoie est professeur agrégé au Département de mathématiques et de statistiques (DMS) de l'Université de Montréal et membre académique principal de Mila – Institut québécois d’intelligence artificielle. Il est titulaire d'une chaire CIFAR (CCAI Canada) ainsi que d'une chaire de recherche du Canada (CRC) en calcul et interfaçage neuronaux.

Ses recherches sont positionnées à l'intersection de l'IA et des neurosciences où il développe des outils pour mieux comprendre les mécanismes d'intelligence communs aux systèmes biologiques et artificiels. Les contributions de son groupe de recherche vont des progrès des paradigmes d'apprentissage à plusieurs échelles pour les grands systèmes artificiels aux applications en neurotechnologie. Dr. Lajoie participe activement aux efforts de développement responsables de l'IA, cherchant à identifier les lignes directrices et les meilleures pratiques pour l'utilisation de l'IA dans la recherche et au-delà.

Étudiants actuels

Matteo Alleman

Postdoctorat - UdeM

Rohan Banerjee

Collaborateur·rice alumni - Polytechnique

rohan.banerjee@mila.quebec

Visiteur de recherche indépendant - N/A

philippe.beaudoin@mila.quebec

Doctorat - UdeM

Co-superviseur⋅e :

Hugo Larochelle

sangnie.bhardwaj@mila.quebec

Colin Bredenberg

Visiteur de recherche indépendant - UdeM

Co-superviseur⋅e :

colin.bredenberg@mila.quebec

Leo Choiniere

Doctorat - UdeM

leo.choiniere@mila.quebec

Claudia Clopath

Visiteur de recherche indépendant - University College London

claudia.clopath@mila.quebec

Eric Elmoznino

Doctorat - UdeM

Superviseur⋅e principal⋅e :

Yoshua Bengio

eric.elmoznino@mila.quebec

Leo Gagnon

Doctorat - UdeM

leo.gagnon@mila.quebec

Tom George

Postdoctorat - McGill

Superviseur⋅e principal⋅e :

tomgeorge1@btinternet.com

Juan Guerra

Doctorat - Polytechnique

juan.guerra@mila.quebec

thomas.jiralerspong@mila.quebec

Nanda Harishankar Krishna

Doctorat - UdeM

Anna Jahn

Visiteur de recherche indépendant - McGill

anna.jahn1@mila.quebec

Chen Jiang

Doctorat - McGill

Superviseur⋅e principal⋅e :

Paul Masset

chen.jiang@mila.quebec

Thomas Jiralerspong

Doctorat - UdeM

Co-superviseur⋅e :

Yoshua Bengio

Maîtrise recherche - UdeM

Co-superviseur⋅e :

Dhanya Sridhar

tejaskasetty@gmail.com

Doctorat - McGill

Superviseur⋅e principal⋅e :

pingsheng.li@mila.quebec

mathys.loiselle@mila.quebec

Mathys Loiselle

Stagiaire de recherche - Concordia

Co-superviseur⋅e :

Matt Perich

Ximeng Mao

Doctorat - UdeM

Co-superviseur⋅e :

Joelle Pineau

ximeng.mao@mila.quebec

Abdel Mfougouon Njupoun

Doctorat - UdeM

Co-superviseur⋅e :

alexandre.payeur@mila.quebec

abdelnjupoun@gmail.com

Doctorat - UdeM

Co-superviseur⋅e :

Visiteur de recherche indépendant - UdeM

Julia Price

Maîtrise recherche - UdeM

julia.price@mila.quebec

Mauricio Rivera

Maîtrise recherche - UdeM

Superviseur⋅e principal⋅e :

Marco Bonizzato

mauriv2010@gmail.com

hee-woon.ryoo@mila.quebec

Avery Ryoo

Doctorat - UdeM

Superviseur⋅e principal⋅e :

Matt Perich

Doctorat - UdeM

Co-superviseur⋅e :

Lune Bellec

sainathp@mila.quebec

ezekiel.williams@mila.quebec

Ryan Vogt

Postdoctorat - UdeM

ryan.vogt@mila.quebec

Ezekiel Williams

Doctorat - UdeM

Visiteur de recherche indépendant - University of Western Ontario

jinkang.xiang@mila.quebec

Billets de blogue

Représentation graphique d'un nerf vague

21 mai 2025

Apprentissage automatique pour la segmentation des différentes activations des fibres nerveuses à partir des signaux neuronaux du cerveau vers le corps

par

Param Raval

Olivier Tessier-Larivière

Pascal Fortier-Poisson

Blake Richards

Guillaume Lajoie

Lire l'article

13 juin 2024

Que nous apprennent les distributions des coefficients synaptiques au sujet de l’apprentissage dans le cerveau ?

par

Roman Pogodin

Jonathan Cornford

Arna Ghosh

Gauthier Gidel

Guillaume Lajoie

Blake Richards

Lire l'article

Publications

Implicit Regularization via Neural Feature Alignment

R Devon Hjelm

We approach the problem of implicit regularization in deep learning from a geometrical viewpoint. We highlight a regularization effect induc… (voir plus)ed by a dynamical alignment of the neural tangent features introduced by Jacot et al, along a small number of task-relevant directions. This can be interpreted as a combined mechanism of feature selection and compression. By extrapolating a new analysis of Rademacher complexity bounds for linear models, we motivate and study a heuristic complexity measure that captures this phenomenon, in terms of sequences of tangent kernel classes along optimization paths.

2021-03-17

International Conference on Artificial Intelligence and Statistics (inconnu)

proceedings.mlr.press

Untangling tradeoffs between recurrence and self-attention in neural networks

2020-12-06

Advances in Neural Information Processing Systems (Accept (poster))

arxiv.org

Learning to Combine Top-Down and Bottom-Up Signals in Recurrent Neural Networks with Attention over Modules

Murray Shanahan

Michael Mozer

Robust perception relies on both bottom-up and top-down signals. Bottom-up signals consist of what's directly observed through sensation. To… (voir plus)p-down signals consist of beliefs and expectations based on past experience and short-term memory, such as how the phrase `peanut butter and~...' will be completed. The optimal combination of bottom-up and top-down information remains an open question, but the manner of combination must be dynamic and both context and task dependent. To effectively utilize the wealth of potential top-down information available, and to prevent the cacophony of intermixed signals in a bidirectional architecture, mechanisms are needed to restrict information flow. We explore deep recurrent neural net architectures in which bottom-up and top-down signals are dynamically combined using attention. Modularity of the architecture further restricts the sharing and communication of information. Together, attention and modularity direct information flow, which leads to reliable performance improvements in perceptual and language tasks, and in particular improves robustness to distractions and noisy data. We demonstrate on a variety of benchmarks in language modeling, sequential image classification, video prediction and reinforcement learning that the \emph{bidirectional} information flow can improve results over strong baselines.

2020-11-20

Proceedings of the 37th International Conference on Machine Learning (publié)

proceedings.mlr.press

Advantages of biologically-inspired adaptive neural activation in RNNs during learning

Dynamic adaptation in single-neuron response plays a fundamental role in neural coding in biological neural networks. Yet, most neural activ… (voir plus)ation functions used in artificial networks are fixed and mostly considered as an inconsequential architecture choice. In this paper, we investigate nonlinear activation function adaptation over the large time scale of learning, and outline its impact on sequential processing in recurrent neural networks. We introduce a novel parametric family of nonlinear activation functions, inspired by input-frequency response curves of biological neurons, which allows interpolation between well-known activation functions such as ReLU and sigmoid. Using simple numerical experiments and tools from dynamical systems and information theory, we study the role of neural activation features in learning dynamics. We find that activation adaptation provides distinct task-specific solutions and in some cases, improves both learning speed and performance. Importantly, we find that optimal activation features emerging from our parametric family are considerably different from typical functions used in the literature, suggesting that exploiting the gap between these usual configurations can help learning. Finally, we outline situations where neural activation adaptation alone may help mitigate changes in input statistics in a given task, suggesting mechanisms for transfer learning optimization.

2020-06-21

ArXiv (prépublication)

arxiv.org

Learning Long-term Dependencies Using Cognitive Inductive Biases in Self-attention RNNs

Attention and self-attention mechanisms, inspired by cognitive processes, are now central to state-of-the-art deep learning on sequential ta… (voir plus)sks. However, most recent progress hinges on heuristic approaches that rely on considerable memory and computational resources that scale poorly. In this work, we propose a relevancy screening mechanism, inspired by the cognitive process of memory consolidation, that allows for a scalable use of sparse self-attention with recurrence. We use simple numerical experiments to demonstrate that this mechanism helps enable recurrent systems on generalization and transfer learning tasks. Based on our results, we propose a concrete direction of research to improve scalability and generalization of attentive recurrent networks.

2019-12-31

(publié)

www.semanticscholar.org

Learning to Optimize with Recurrent Hierarchical Transformers

2019-12-31

International Conference on Machine Learning (publié)

openreview.net

Non-normal Recurrent Neural Network (nnRNN): learning long time dependencies while improving expressivity with transient dynamics

Giancarlo Kerg

Kyle Goyette

Maximilian Puelma Touzel

A recent strategy to circumvent the exploding and vanishing gradient problem in RNNs, and to allow the stable propagation of signals over lo… (voir plus)ng time scales, is to constrain recurrent connectivity matrices to be orthogonal or unitary. This ensures eigenvalues with unit norm and thus stable dynamics and training. However this comes at the cost of reduced expressivity due to the limited variety of orthogonal transformations. We propose a novel connectivity structure based on the Schur decomposition and a splitting of the Schur form into normal and non-normal parts. This allows to parametrize matrices with unit-norm eigenspectra without orthogonality constraints on eigenbases. The resulting architecture ensures access to a larger space of spectrally constrained matrices, of which orthogonal matrices are a subset. This crucial difference retains the stability advantages and training speed of orthogonal RNNs while enhancing expressivity, especially on tasks that require computations over ongoing input sequences.

2019-05-27

PolyPublie (accepté)