Guillaume Lajoie

alexandre.payeur@mila.quebec

Biography

Guillaume Lajoie is an assistant professor in the Department of Mathematics and Statistics at Université de Montréal and a core academic member of Mila – Quebec Artificial Intelligence Institute. He is also a Fonds de recherche du Québec - Health Research Scholar and holds a Tier 2 Canada Research Chair in Neural Computation and Interfacing.

Previously, Lajoie was a postdoctoral fellow at the Max Planck Institute for Dynamics and Self-Organization in Germany and at the University of Washington’s Institute for Neuroengineering. He obtained his PhD from the Department of Applied Mathematics at the University of Washington (Seattle).

Lying at the intersection of AI and neuroscience, Lajoie’s research pursues questions surrounding neural network dynamics and computations, which has potential applications to neuroengineering.

Recent work has focused on the development of architectural inductive biases for information propagation in recurrent networks, as well as the development of algorithms and models for bidirectional brain-machine interface optimization.

Current Students

Alexandre Payeur

Collaborating researcher - Université de Montréal

Amine Natik

PhD - Université de Montréal

Co-supervisor :

natikami@mila.quebec

Colin Bredenberg

Postdoctorate - Université de Montréal

Co-supervisor :

colin.bredenberg@mila.quebec

Eric Elmoznino

PhD - Université de Montréal

Principal supervisor :

Yoshua Bengio

eric.elmoznino@mila.quebec

PhD - Université de Montréal

ezekiel.williams@mila.quebec

PhD - Université de Montréal

paugamfr@mila.quebec

hee-woon.ryoo@mila.quebec

Avery Ryoo

Master's Research - Université de Montréal

Principal supervisor :

Matt Perich

Website

yuhan-helena.liu@mila.quebec

Helena Yuhan Liu

Collaborating researcher - Université de Montréal

jean-pierre.falet@mila.quebec

Jean-pierre Falet

PhD - Université de Montréal

Principal supervisor :

Yoshua Bengio

Website

Juan Guerra

Master's Research - Polytechnique Montréal

Principal supervisor :

juan.guerra@mila.quebec

Laura Suarez

PhD - McGill University

suarezul@mila.quebec

Leo Gagnon

PhD - Université de Montréal

leo.gagnon@mila.quebec

leo.choiniere@mila.quebec

Leo Choiniere

PhD - Université de Montréal

Nanda Harishankar Krishna

PhD - Université de Montréal

nanda.harishankar-krishna@mila.quebec

Postdoctorate - Université de Montréal

Co-supervisor :

Matt Perich

olivier.codol@mila.quebec

Pravish Sainath

PhD - Université de Montréal

Co-supervisor :

Pierre (Louis) Bellec

sainathp@mila.quebec

Roman Pogodin

Postdoctorate - McGill University

Principal supervisor :

Roman.Pogodin@mila.Quebec

Master's Research - Polytechnique Montréal

Principal supervisor :

PhD - Université de Montréal

Co-supervisor :

Hugo Larochelle

sangnie.bhardwaj@mila.quebec

Sarthak Mittal

PhD - Université de Montréal

Co-supervisor :

Independent visiting researcher

Principal supervisor :

Yoshua Bengio

stefan.bauer@mila.quebec

tejas.kasetty@mila.quebec

Tejas Kasetty

Professional Master's - Université de Montréal

Collaborating researcher - Polytechnique Montréal

Principal supervisor :

thomas.garbay@mila.quebec

Vivian White

Research Intern - Western Washington University

Co-supervisor :

vivian.white@mila.quebec

Ximeng Mao

PhD - Université de Montréal

Co-supervisor :

Joelle Pineau

ximeng.mao@mila.quebec

What Do Synaptic Weight Distributions Tell Us About Learning in the Brain ?

Blog Posts

June 13, 2024

Roman Pogodin

Jonathan Cornford

Arna Ghosh

Gauthier Gidel

Guillaume Lajoie

Blake Richards

Read the article

Publications

LEAD: Min-Max Optimization from a Physical Perspective

Reyhane Askari Hemmat

Amartya Mitra

Ioannis Mitliagkas

Adversarial formulations have rekindled interest in two-player min-max games. A central obstacle in the optimization of such games is the ro… (see more)tational dynamics that hinder their convergence. In this paper, we show that game optimization shares dynamic properties with particle systems subject to multiple forces, and one can leverage tools from physics to improve optimization dynamics. Inspired by the physical framework, we propose LEAD, an optimizer for min-max games. Next, using Lyapunov stability theory from dynamical systems as well as spectral analysis, we study LEAD’s convergence properties in continuous and discrete time settings for a class of quadratic min-max games to demonstrate linear convergence to the Nash equilibrium. Finally, we empirically evaluate our method on synthetic setups and CIFAR-10 image generation to demonstrate improvements in GAN training.

2023-06-15

TMLR (accepted)

conn2res: A toolbox for connectome-based reservoir computing

Laura E. Suárez

Agoston Mihalik

Filip Milisav

Kenji Marshall

Mingze Li

Petra E. Vértes

Bratislav Mišić

2023-06-04

bioRxiv (preprint)

Autonomous optimization of neuroprosthetic stimulation parameters that drive the motor cortex and spinal cord outputs in rats and monkeys

Rose Guay Hottin

Sandrine L. Côté

Elena Massai

Léo Choinière

Uzay Macar

Samuel Laferrière

Parikshat Sirpal

Stephan Quessy

Marina Martinez

Numa Dancause

2023-04-11

Cell Reports Medicine (published)

Multi-view manifold learning of human brain state trajectories

Erica Lindsey Busch

Je-chun Huang

Andrew Benz

Tom Wallenstein

Smita Krishnaswamy

Nicholas Turk-Browne

2023-03-27

Nature Computational Science (published)

Transfer Entropy Bottleneck: Learning Sequence to Sequence Information Transfer

Damjan Kalajdzievski

Ximeng Mao

Pascal Fortier-Poisson

When presented with a data stream of two statistically dependent variables, predicting the future of one of the variables (the target stream… (see more)) can benefit from information about both its history and the history of the other variable (the source stream). For example, fluctuations in temperature at a weather station can be predicted using both temperatures and barometric readings. However, a challenge when modelling such data is that it is easy for a neural network to rely on the greatest joint correlations within the target stream, which may ignore a crucial but small information transfer from the source to the target stream. As well, there are often situations where the target stream may have previously been modelled independently and it would be useful to use that model to inform a new joint model. Here, we develop an information bottleneck approach for conditional learning on two dependent streams of data. Our method, which we call Transfer Entropy Bottleneck (TEB), allows one to learn a model that bottlenecks the directed information transferred from the source variable to the target variable, while quantifying this information transfer within the model. As such, TEB provides a useful new information bottleneck approach for modelling two statistically dependent streams of data in order to make predictions about one of them.

2023-03-08

TMLR (accepted)

Use of Invasive Brain-Computer Interfaces in Pediatric Neurosurgery: Technical and Ethical Considerations

David Bergeron

Christian Iorio-Morin

Nathalie Orr Gaucher

Éric Racine

Alexander G. Weil

2023-03-01

Journal of Child Neurology (published)

Steerable Equivariant Representation Learning

Sangnie Bhardwaj

Willie McClinton

Tongzhou Wang

Chen Sun

Phillip Isola

Dilip Krishnan

Pre-trained deep image representations are useful for post-training tasks such as classification through transfer learning, image retrieval,… (see more) and object detection. Data augmentations are a crucial aspect of pre-training robust representations in both supervised and self-supervised settings. Data augmentations explicitly or implicitly promote invariance in the embedding space to the input image transformations. This invariance reduces generalization to those downstream tasks which rely on sensitivity to these particular data augmentations. In this paper, we propose a method of learning representations that are instead equivariant to data augmentations. We achieve this equivariance through the use of steerable representations. Our representations can be manipulated directly in embedding space via learned linear maps. We demonstrate that our resulting steerable and equivariant representations lead to better performance on transfer learning and robustness: e.g. we improve linear probe top-1 accuracy by between 1% to 3% for transfer; and ImageNet-C accuracy by upto 3.4%. We further show that the steerability of our representations provides significant speedup (nearly 50x) for test-time augmentations; by applying a large number of augmentations for out-of-distribution detection, we significantly improve OOD AUC on the ImageNet-C dataset over an invariant representation.

2023-02-22

ArXiv (preprint)

How gradient estimator variance and bias impact learning in neural networks

Arna Ghosh

Yuhan Helena Liu

Konrad Paul Kording

There is growing interest in understanding how real brains may approximate gradients and how gradients can be used to train neuromorphic chi… (see more)ps. However, neither real brains nor neuromorphic chips can perfectly follow the loss gradient, so parameter updates would necessarily use gradient estimators that have some variance and/or bias. Therefore, there is a need to understand better how variance and bias in gradient estimators impact learning dependent on network and task properties. Here, we show that variance and bias can impair learning on the training data, but some degree of variance and bias in a gradient estimator can be beneficial for generalization. We find that the ideal amount of variance and bias in a gradient estimator are dependent on several properties of the network and task: the size and activity sparsity of the network, the norm of the gradient, and the curvature of the loss landscape. As such, whether considering biologically-plausible learning algorithms or algorithms for training neuromorphic chips, researchers can analyze these properties to determine whether their approximation to gradient descent will be effective for learning given their network and task properties.

2023-02-01

ICLR.cc/2023/Conference (poster)

Reliability of CKA as a Similarity Measure in Deep Learning

MohammadReza Davari

Stefan Horoi

Amine Natik

Eugene Belilovsky

Comparing learned neural representations in neural networks is a challenging but important problem, which has been approached in different w… (see more)ays. The Centered Kernel Alignment (CKA) similarity metric, particularly its linear variant, has recently become a popular approach and has been widely used to compare representations of a network's different layers, of architecturally similar networks trained differently, or of models with different architectures trained on the same data. A wide variety of claims about similarity and dissimilarity of these various representations have been made using CKA results. In this work we present analysis that formally characterizes CKA sensitivity to a large class of simple transformations, which can naturally occur in the context of modern machine learning. This provides a concrete explanation to CKA sensitivity to outliers, which has been observed in past works, and to transformations that preserve the linear separability of the data, an important generalization attribute. We empirically investigate several weaknesses of the CKA similarity metric, demonstrating situations in which it gives unexpected or counterintuitive results. Finally we study approaches for modifying representations to maintain functional behaviour while changing the CKA value. Our results illustrate that, in many cases, the CKA value can be easily manipulated without substantial changes to the functional behaviour of the models, and call for caution when leveraging activation alignment metrics.

2023-02-01

ICLR.cc/2023/Conference (poster)

« Que notre cerveau soit constitué de neurones n’est pas un accident »

Roman Ikonicoff

2023-01-02

Pour la science (published)