Guillaume Lajoie

alexandre.payeur@mila.quebec

Biography

Guillaume Lajoie is an assistant professor in the Department of Mathematics and Statistics at Université de Montréal and a core academic member of Mila – Quebec Artificial Intelligence Institute. He is also a Fonds de recherche du Québec - Health Research Scholar and holds a Tier 2 Canada Research Chair in Neural Computation and Interfacing.

Previously, Lajoie was a postdoctoral fellow at the Max Planck Institute for Dynamics and Self-Organization in Germany and at the University of Washington’s Institute for Neuroengineering. He obtained his PhD from the Department of Applied Mathematics at the University of Washington (Seattle).

Lying at the intersection of AI and neuroscience, Lajoie’s research pursues questions surrounding neural network dynamics and computations, which has potential applications to neuroengineering.

Recent work has focused on the development of architectural inductive biases for information propagation in recurrent networks, as well as the development of algorithms and models for bidirectional brain-machine interface optimization.

Current Students

Alexandre Payeur

Collaborating researcher - Université de Montréal

Amine Natik

PhD - Université de Montréal

Co-supervisor :

Guy Wolf

natikami@mila.quebec

Colin Bredenberg

Postdoctorate - Université de Montréal

Co-supervisor :

colin.bredenberg@mila.quebec

Eric Elmoznino

PhD - Université de Montréal

Principal supervisor :

eric.elmoznino@mila.quebec

PhD - Université de Montréal

ezekiel.williams@mila.quebec

PhD - Université de Montréal

paugamfr@mila.quebec

hee-woon.ryoo@mila.quebec

Avery Ryoo

Master's Research - Université de Montréal

Principal supervisor :

Matt Perich

Website

yuhan-helena.liu@mila.quebec

Helena Yuhan Liu

Collaborating researcher - Université de Montréal

Jean-pierre Falet

PhD - Université de Montréal

Principal supervisor :

jean-pierre.falet@mila.quebec

Website

Juan Guerra

Master's Research - Polytechnique Montréal

Principal supervisor :

Marco Bonizzato

juan.guerra@mila.quebec

Laura Suarez

PhD - McGill University

suarezul@mila.quebec

Leo Gagnon

PhD - Université de Montréal

leo.gagnon@mila.quebec

leo.choiniere@mila.quebec

Leo Choiniere

PhD - Université de Montréal

Nanda Harishankar Krishna

PhD - Université de Montréal

nanda.harishankar-krishna@mila.quebec

Postdoctorate - Université de Montréal

Co-supervisor :

Matt Perich

olivier.codol@mila.quebec

Pravish Sainath

PhD - Université de Montréal

Co-supervisor :

Pierre (Louis) Bellec

sainathp@mila.quebec

Roman Pogodin

Postdoctorate - McGill University

Principal supervisor :

Roman.Pogodin@mila.Quebec

Master's Research - Polytechnique Montréal

Principal supervisor :

PhD - Université de Montréal

Co-supervisor :

Hugo Larochelle

sangnie.bhardwaj@mila.quebec

Sarthak Mittal

PhD - Université de Montréal

Co-supervisor :

Independent visiting researcher

Principal supervisor :

stefan.bauer@mila.quebec

tejas.kasetty@mila.quebec

Tejas Kasetty

Professional Master's - Université de Montréal

Collaborating researcher - Polytechnique Montréal

Principal supervisor :

Marco Bonizzato

thomas.garbay@mila.quebec

Vivian White

Research Intern - Western Washington University

Co-supervisor :

Guy Wolf

vivian.white@mila.quebec

Ximeng Mao

PhD - Université de Montréal

Co-supervisor :

Joelle Pineau

ximeng.mao@mila.quebec

What Do Synaptic Weight Distributions Tell Us About Learning in the Brain ?

Blog Posts

June 13, 2024

Roman Pogodin

Jonathan Cornford

Arna Ghosh

Gauthier Gidel

Guillaume Lajoie

Blake Richards

Read the article

Publications

Sufficient conditions for offline reactivation in recurrent neural networks

Nanda H Krishna

Colin Bredenberg

Daniel Levenstein

During periods of quiescence, such as sleep, neural activity in many brain circuits resembles that observed during periods of task engagemen… (see more)t. However, the precise conditions under which task-optimized networks can autonomously reactivate the same network states responsible for online behavior is poorly understood. In this study, we develop a mathematical framework that outlines sufficient conditions for the emergence of neural reactivation in circuits that encode features of smoothly varying stimuli. We demonstrate mathematically that noisy recurrent networks optimized to track environmental state variables using change-based sensory information naturally develop denoising dynamics, which, in the absence of input, cause the network to revisit state configurations observed during periods of online activity. We validate our findings using numerical experiments on two canonical neuroscience tasks: spatial position estimation based on self-motion cues, and head direction estimation based on angular velocity cues. Overall, our work provides theoretical support for modeling offline reactivation as an emergent consequence of task optimization in noisy neural circuits.

2024-01-16

ICLR.cc/2024/Conference (poster)

Synaptic Weight Distributions Depend on the Geometry of Plasticity

Roman Pogodin

Jonathan Cornford

Arna Ghosh

Gauthier Gidel

A growing literature in computational neuroscience leverages gradient descent and learning algorithms that approximate it to study synaptic … (see more)plasticity in the brain. However, the vast majority of this work ignores a critical underlying assumption: the choice of distance for synaptic changes - i.e. the geometry of synaptic plasticity. Gradient descent assumes that the distance is Euclidean, but many other distances are possible, and there is no reason that biology necessarily uses Euclidean geometry. Here, using the theoretical tools provided by mirror descent, we show that the distribution of synaptic weights will depend on the geometry of synaptic plasticity. We use these results to show that experimentally-observed log-normal weight distributions found in several brain areas are not consistent with standard gradient descent (i.e. a Euclidean geometry), but rather with non-Euclidean distances. Finally, we show that it should be possible to experimentally test for different synaptic geometries by comparing synaptic weight distributions before and after learning. Overall, our work shows that the current paradigm in theoretical work on synaptic plasticity that assumes Euclidean synaptic geometry may be misguided and that it should be possible to experimentally determine the true geometry of synaptic plasticity in the brain.

2024-01-16

ICLR.cc/2024/Conference (spotlight)

Personalized inference for neurostimulation with meta-learning: a case study of vagus nerve stimulation

Ximeng Mao

Yao-Chuan Chang

Stavros Zanos

2024-01-12

Journal of Neural Engineering (published)

Neural manifolds and learning regimes in neural-interface tasks

Alexandre Payeur

Amy L. Orsborn

2023-12-23

bioRxiv (preprint)

Large language models: What could they do for neurology?

2023-12-01

Journal of the Neurological Sciences (published)

Discrete, compositional, and symbolic representations through attractor dynamics

Andrew Nam

Eric Elmoznino

Nikolay Malkin

Chen Sun

Compositionality is an important feature of discrete symbolic systems, such as language and programs, as it enables them to have infinite ca… (see more)pacity despite a finite symbol set. It serves as a useful abstraction for reasoning in both cognitive science and in AI, yet the interface between continuous and symbolic processing is often imposed by fiat at the algorithmic level, such as by means of quantization or a softmax sampling step. In this work, we explore how discretization could be implemented in a more neurally plausible manner through the modeling of attractor dynamics that partition the continuous representation space into basins that correspond to sequences of symbols. Building on established work in attractor networks and introducing novel training methods, we show that imposing structure in the symbolic space can produce compositionality in the attractor-supported representation space of rich sensory inputs. Lastly, we argue that our model exhibits the process of an information bottleneck that is thought to play a role in conscious experience, decomposing the rich information of a sensory input into stable components encoding symbolic information.

2023-10-27

NeurIPS.cc/2023/Workshop/InfoCog (oral)

A Unified, Scalable Framework for Neural Population Decoding

Mehdi Azabou

Vinam Arora

Venkataramana Ganesh

Ximeng Mao

Santosh B Nachimuthu

Michael Jacob Mendelson

Matt Perich

Eva L Dyer

Our ability to use deep learning approaches to decipher neural activity would likely benefit from greater scale, in terms of both the model … (see more)size and the datasets. However, the integration of many neural recordings into one unified model is challenging, as each recording contains the activity of different neurons from different individual animals. In this paper, we introduce a training framework and architecture designed to model the population dynamics of neural activity across diverse, large-scale neural recordings. Our method first tokenizes individual spikes within the dataset to build an efficient representation of neural events that captures the fine temporal structure of neural activity. We then employ cross-attention and a PerceiverIO backbone to further construct a latent tokenization of neural population activities. Utilizing this architecture and training framework, we construct a large-scale multi-session model trained on large datasets from seven nonhuman primates, spanning over 158 different sessions of recording from over 27,373 neural units and over 100 hours of recordings. In a number of different tasks, we demonstrate that our pretrained model can be rapidly adapted to new, unseen sessions with unspecified neuron correspondence, enabling few-shot performance with minimal labels. This work presents a powerful new approach for building deep learning tools to analyze neural data and stakes out a clear path to training at scale for neural decoding models.

Online Bayesian Optimization of Nerve Stimulation

Lorenz Wernisch

Tristan Edwards

Antonin Berthon

Olivier Tessier-Lariviere

Elvijs Sarkans

Myrta Stoukidi

Pascal Fortier-Poisson

Max Pinkney

Michael Thornton

Catherine Hanley

Susannah Lee

Joel Jennings

Ben Appleton

Philip Garsed

Bret Patterson

Buttinger Will

Samuel Gonshaw

Matjaž Jakopec

Sudhakaran Shunmugam

Jorin Mamen … (see 4 more)

Aleksi Tukiainen

Oliver Armitage

Emil Hewage

2023-09-01

bioRxiv (preprint)

Neural networks with optimized single-neuron adaptation uncover biologically plausible regularization

Victor Geadah

Stefan Horoi

Giancarlo Kerg

Guy Wolf

Neurons in the brain have rich and adaptive input-output properties. Features such as heterogeneous f-I curves and spike frequency adaptatio… (see more)n are known to place single neurons in optimal coding regimes when facing changing stimuli. Yet, it is still unclear how brain circuits exploit single-neuron flexibility, and how network-level requirements may have shaped such cellular function. To answer this question, a multi-scaled approach is needed where the computations of single neurons and neural circuits must be considered as a complete system. In this work, we use artificial neural networks to systematically investigate single-neuron input-output adaptive mechanisms, optimized in an end-to-end fashion. Throughout the optimization process, each neuron has the liberty to modify its nonlinear activation function, parametrized to mimic f-I curves of biological neurons, and to learn adaptation strategies to modify activation functions in real-time during a task. We find that such networks show much-improved robustness to noise and changes in input statistics. Importantly, we find that this procedure recovers precise coding strategies found in biological neurons, such as gain scaling and fractional order differentiation/integration. Using tools from dynamical systems theory, we analyze the role of these emergent single-neuron properties and argue that neural diversity and adaptation play an active regularization role, enabling neural circuits to optimally propagate information across time.

2023-07-19

bioRxiv (preprint)

Flexible Phase Dynamics for Bio-Plausible Contrastive Learning

Ezekiel Williams

Colin Bredenberg

2023-07-03

Proceedings of the 40th International Conference on Machine Learning (published)

Exploring Exchangeable Dataset Amortization for Bayesian Posterior Inference

Sarthak Mittal

Niels Leif Bracher

Priyank Jaini

Marcus A Brubaker

Bayesian inference provides a natural way of incorporating uncertainties and different underlying theories when making predictions or analyz… (see more)ing complex systems. However, it requires computationally expensive routines for approximation, which have to be re-run when new data is observed and are thus infeasible to efficiently scale and reuse. In this work, we look at the problem from the perspective of amortized inference to obtain posterior parameter distributions for known probabilistic models. We propose a neural network-based approach that can handle exchangeable observations and amortize over datasets to convert the problem of Bayesian posterior inference into a single forward pass of a network. Our empirical analyses explore various design choices for amortized inference by comparing: (a) our proposed variational objective with forward KL minimization, (b) permutation-invariant architectures like Transformers and DeepSets, and (c) parameterizations of posterior families like diagonal Gaussian and Normalizing Flows. Through our experiments, we successfully apply amortization techniques to estimate the posterior distributions for different domains solely through inference.

2023-06-19

ICML.cc/2023/Workshop/SPIGM (poster)

Learning to Optimize with Recurrent Hierarchical Transformers

Abhinav Moudgil

Boris Knyazev

Eugene Belilovsky

2023-06-19

ICML.cc/2023/Workshop/Frontiers4LCD (published)