Guillaume Lajoie

Biography

Guillaume Lajoie is an Associate professor in the Department of Mathematics and Statistics at Université de Montréal and a Core Academic Member of Mila – Quebec Artificial Intelligence Institute. He holds a Canada-CIFAR AI Research Chair, and a Canada Research Chair (CRC) in Neural Computation and Interfacing. He also holds a Health Research Scholar of Fonds de recherche du Québec.

Guillaume Lajoie was previously a postdoctoral fellow at the Max Planck Institute for Dynamics and Self-Organization in Germany and at the University of Washington’s Institute for Neuroengineering. He obtained his PhD from the Department of Applied Mathematics at the University of Washington (Seattle).

His research is positioned at the intersection of AI and Neuroscience where he develops tools to better understand mechanisms of intelligence common to both biological and artificial systems. His research group's contributions range from advances in multi-scale learning paradigms for large artificial systems, to applications in neurotechnology. Dr. Lajoie is actively involved in responsible AI development efforts, seeking to identify guidelines and best practices for use of AI in research and beyond.

Recent work has focused on the development of architectural inductive biases for information propagation in recurrent networks, as well as the development of algorithms and models for bidirectional brain-machine interface optimization.

Current Students

Stefan Bauer

Independent visiting researcher

Principal supervisor :

Yoshua Bengio

Sangnie Bhardwaj

PhD - Université de Montréal

Co-supervisor :

Hugo Larochelle

Colin Bredenberg

Postdoctorate - Université de Montréal

Co-supervisor :

Blake Richards

Leo Choiniere

PhD - Université de Montréal

Olivier Codol

Postdoctorate - Université de Montréal

Co-supervisor :

Matt Perich

Eric Elmoznino

PhD - Université de Montréal

Principal supervisor :

PhD - Université de Montréal

Principal supervisor :

Leo Gagnon

PhD - Université de Montréal

Rose Guay Hottin

Master's Research - Polytechnique Montréal

Principal supervisor :

Juan Guerra

Master's Research - Polytechnique Montréal

Principal supervisor :

Website

Nanda Harishankar Krishna

PhD - Université de Montréal

Collaborating researcher - Western Washington University (faculty; assistant prof))

Principal supervisor :

Master's Research - Université de Montréal

Co-supervisor :

Collaborating researcher - Université de Montréal

Ximeng Mao

PhD - Université de Montréal

Co-supervisor :

Joelle Pineau

Abdel Mfougouon Njupoun

PhD - Université de Montréal

Co-supervisor :

PhD - Université de Montréal

Co-supervisor :

Amine Natik

PhD - Université de Montréal

Co-supervisor :

Guy Wolf

François Paugam

PhD - Université de Montréal

Principal supervisor :

Lune Bellec

Alexandre Payeur

Collaborating researcher - Université de Montréal

Roman Pogodin

Postdoctorate - McGill University

Principal supervisor :

Avery Ryoo

Master's Research - Université de Montréal

Principal supervisor :

PhD - Université de Montréal

Co-supervisor :

Lune Bellec

Laura Suarez

PhD - McGill University

Vivian White

Research Intern - Western Washington University

Co-supervisor :

PhD - Université de Montréal

Website

What Do Synaptic Weight Distributions Tell Us About Learning in the Brain ?

Blog Posts

June 13, 2024

Roman Pogodin

Jonathan Cornford

Arna Ghosh

Gauthier Gidel

Guillaume Lajoie

Blake Richards

Read the article

Publications

Learning to Optimize with Recurrent Hierarchical Transformers

Abhinav Moudgil

Boris Knyazev

Eugene Belilovsky

2023-06-19

ICML.cc/2023/Workshop/Frontiers4LCD (published)

conn2res: A toolbox for connectome-based reservoir computing

Laura E. Suárez

Agoston Mihalik

Filip Milisav

Kenji Marshall

Mingze Li

Petra E. Vértes

Bratislav Mišić

2023-06-04

bioRxiv (preprint)

Autonomous optimization of neuroprosthetic stimulation parameters that drive the motor cortex and spinal cord outputs in rats and monkeys

Rose Guay Hottin

Sandrine L. Côté

Elena Massai

Léo Choinière

Uzay Macar

Samuel Laferrière

Parikshat Sirpal

Stephan Quessy

Marina Martinez

Numa Dancause

2023-04-11

Cell Reports Medicine (published)

Multi-view manifold learning of human brain state trajectories

Erica Lindsey Busch

Je-chun Huang

Andrew Benz

Tom Wallenstein

Guy Wolf

Smita Krishnaswamy

Nicholas Turk-Browne

2023-03-27

Nature Computational Science (published)

Transfer Entropy Bottleneck: Learning Sequence to Sequence Information Transfer

Damjan Kalajdzievski

Ximeng Mao

Pascal Fortier-Poisson

Blake Richards

When presented with a data stream of two statistically dependent variables, predicting the future of one of the variables (the target stream… (see more)) can benefit from information about both its history and the history of the other variable (the source stream). For example, fluctuations in temperature at a weather station can be predicted using both temperatures and barometric readings. However, a challenge when modelling such data is that it is easy for a neural network to rely on the greatest joint correlations within the target stream, which may ignore a crucial but small information transfer from the source to the target stream. As well, there are often situations where the target stream may have previously been modelled independently and it would be useful to use that model to inform a new joint model. Here, we develop an information bottleneck approach for conditional learning on two dependent streams of data. Our method, which we call Transfer Entropy Bottleneck (TEB), allows one to learn a model that bottlenecks the directed information transferred from the source variable to the target variable, while quantifying this information transfer within the model. As such, TEB provides a useful new information bottleneck approach for modelling two statistically dependent streams of data in order to make predictions about one of them.

2023-03-08

TMLR (accepted)

Use of Invasive Brain-Computer Interfaces in Pediatric Neurosurgery: Technical and Ethical Considerations

David Bergeron

Christian Iorio-Morin

Nathalie Orr Gaucher

Éric Racine

Alexander G. Weil

2023-03-01

Journal of Child Neurology (published)

Steerable Equivariant Representation Learning

Sangnie Bhardwaj

Willie McClinton

Tongzhou Wang

Chen Sun

Phillip Isola

Dilip Krishnan

Pre-trained deep image representations are useful for post-training tasks such as classification through transfer learning, image retrieval,… (see more) and object detection. Data augmentations are a crucial aspect of pre-training robust representations in both supervised and self-supervised settings. Data augmentations explicitly or implicitly promote invariance in the embedding space to the input image transformations. This invariance reduces generalization to those downstream tasks which rely on sensitivity to these particular data augmentations. In this paper, we propose a method of learning representations that are instead equivariant to data augmentations. We achieve this equivariance through the use of steerable representations. Our representations can be manipulated directly in embedding space via learned linear maps. We demonstrate that our resulting steerable and equivariant representations lead to better performance on transfer learning and robustness: e.g. we improve linear probe top-1 accuracy by between 1% to 3% for transfer; and ImageNet-C accuracy by upto 3.4%. We further show that the steerability of our representations provides significant speedup (nearly 50x) for test-time augmentations; by applying a large number of augmentations for out-of-distribution detection, we significantly improve OOD AUC on the ImageNet-C dataset over an invariant representation.

2023-02-22

ArXiv (preprint)

How gradient estimator variance and bias impact learning in neural networks

Arna Ghosh

Yuhan Helena Liu

Konrad Paul Kording

Blake Richards

There is growing interest in understanding how real brains may approximate gradients and how gradients can be used to train neuromorphic chi… (see more)ps. However, neither real brains nor neuromorphic chips can perfectly follow the loss gradient, so parameter updates would necessarily use gradient estimators that have some variance and/or bias. Therefore, there is a need to understand better how variance and bias in gradient estimators impact learning dependent on network and task properties. Here, we show that variance and bias can impair learning on the training data, but some degree of variance and bias in a gradient estimator can be beneficial for generalization. We find that the ideal amount of variance and bias in a gradient estimator are dependent on several properties of the network and task: the size and activity sparsity of the network, the norm of the gradient, and the curvature of the loss landscape. As such, whether considering biologically-plausible learning algorithms or algorithms for training neuromorphic chips, researchers can analyze these properties to determine whether their approximation to gradient descent will be effective for learning given their network and task properties.

2023-02-01

ICLR.cc/2023/Conference (poster)

Reliability of CKA as a Similarity Measure in Deep Learning

MohammadReza Davari

Stefan Horoi

Amine Natik

Guy Wolf

Eugene Belilovsky

Comparing learned neural representations in neural networks is a challenging but important problem, which has been approached in different w… (see more)ays. The Centered Kernel Alignment (CKA) similarity metric, particularly its linear variant, has recently become a popular approach and has been widely used to compare representations of a network's different layers, of architecturally similar networks trained differently, or of models with different architectures trained on the same data. A wide variety of claims about similarity and dissimilarity of these various representations have been made using CKA results. In this work we present analysis that formally characterizes CKA sensitivity to a large class of simple transformations, which can naturally occur in the context of modern machine learning. This provides a concrete explanation to CKA sensitivity to outliers, which has been observed in past works, and to transformations that preserve the linear separability of the data, an important generalization attribute. We empirically investigate several weaknesses of the CKA similarity metric, demonstrating situations in which it gives unexpected or counterintuitive results. Finally we study approaches for modifying representations to maintain functional behaviour while changing the CKA value. Our results illustrate that, in many cases, the CKA value can be easily manipulated without substantial changes to the functional behaviour of the models, and call for caution when leveraging activation alignment metrics.

2023-02-01

ICLR.cc/2023/Conference (poster)

« Que notre cerveau soit constitué de neurones n’est pas un accident »

Roman Ikonicoff

2023-01-02

Pour la science (published)