Guillaume Lajoie

alexandre.payeur@mila.quebec

Biography

Guillaume Lajoie is an assistant professor in the Department of Mathematics and Statistics at Université de Montréal and a core academic member of Mila – Quebec Artificial Intelligence Institute. He is also a Fonds de recherche du Québec - Health Research Scholar and holds a Tier 2 Canada Research Chair in Neural Computation and Interfacing.

Previously, Lajoie was a postdoctoral fellow at the Max Planck Institute for Dynamics and Self-Organization in Germany and at the University of Washington’s Institute for Neuroengineering. He obtained his PhD from the Department of Applied Mathematics at the University of Washington (Seattle).

Lying at the intersection of AI and neuroscience, Lajoie’s research pursues questions surrounding neural network dynamics and computations, which has potential applications to neuroengineering.

Recent work has focused on the development of architectural inductive biases for information propagation in recurrent networks, as well as the development of algorithms and models for bidirectional brain-machine interface optimization.

Current Students

Alexandre Payeur

Collaborating researcher - Université de Montréal

Amine Natik

PhD - Université de Montréal

Co-supervisor :

natikami@mila.quebec

Colin Bredenberg

Postdoctorate - Université de Montréal

Co-supervisor :

colin.bredenberg@mila.quebec

Eric Elmoznino

PhD - Université de Montréal

Principal supervisor :

eric.elmoznino@mila.quebec

PhD - Université de Montréal

ezekiel.williams@mila.quebec

PhD - Université de Montréal

paugamfr@mila.quebec

hee-woon.ryoo@mila.quebec

Avery Ryoo

Master's Research - Université de Montréal

Principal supervisor :

Matt Perich

Website

yuhan-helena.liu@mila.quebec

Helena Yuhan Liu

Collaborating researcher - Université de Montréal

Jean-pierre Falet

PhD - Université de Montréal

Principal supervisor :

jean-pierre.falet@mila.quebec

Website

Juan Guerra

Master's Research - Polytechnique Montréal

Principal supervisor :

Marco Bonizzato

juan.guerra@mila.quebec

Laura Suarez

PhD - McGill University

suarezul@mila.quebec

Leo Gagnon

PhD - Université de Montréal

leo.gagnon@mila.quebec

leo.choiniere@mila.quebec

Leo Choiniere

PhD - Université de Montréal

Nanda Harishankar Krishna

PhD - Université de Montréal

nanda.harishankar-krishna@mila.quebec

Postdoctorate - Université de Montréal

Co-supervisor :

Matt Perich

olivier.codol@mila.quebec

Pravish Sainath

PhD - Université de Montréal

Co-supervisor :

Pierre (Louis) Bellec

sainathp@mila.quebec

Roman Pogodin

Postdoctorate - McGill University

Principal supervisor :

Roman.Pogodin@mila.Quebec

Master's Research - Polytechnique Montréal

Principal supervisor :

PhD - Université de Montréal

Co-supervisor :

Hugo Larochelle

sangnie.bhardwaj@mila.quebec

Sarthak Mittal

PhD - Université de Montréal

Co-supervisor :

Independent visiting researcher

Principal supervisor :

stefan.bauer@mila.quebec

tejas.kasetty@mila.quebec

Tejas Kasetty

Professional Master's - Université de Montréal

Collaborating researcher - Polytechnique Montréal

Principal supervisor :

Marco Bonizzato

thomas.garbay@mila.quebec

Vivian White

Research Intern - Western Washington University

Co-supervisor :

vivian.white@mila.quebec

Ximeng Mao

PhD - Université de Montréal

Co-supervisor :

Joelle Pineau

ximeng.mao@mila.quebec

What Do Synaptic Weight Distributions Tell Us About Learning in the Brain ?

Blog Posts

June 13, 2024

Roman Pogodin

Jonathan Cornford

Arna Ghosh

Gauthier Gidel

Guillaume Lajoie

Blake Richards

Read the article

Publications

A connectomics-based taxonomy of mammals

Laura E. Suárez

Yossi Yovel

Martijn P. van den Heuvel

Olaf Sporns

Yaniv Assaf

Bratislav Mišić

2022-03-12

bioRxiv (preprint)

Beyond accuracy: generalization properties of bio-plausible temporal credit assignment rules

Yuhan Helena Liu

Arna Ghosh

Eric Todd SheaBrown

openreview.net

Compositional Attention: Disentangling Search and Retrieval

Sarthak Mittal

Sharath Chandra Raparthy

Irina Rish

Multi-head, key-value attention is the backbone of transformer-like model architectures which have proven to be widely successful in recent … (see more)years. This attention mechanism uses multiple parallel key-value attention blocks (called heads), each performing two fundamental computations: (1) search - selection of a relevant entity from a set via query-key interaction, and (2) retrieval - extraction of relevant features from the selected entity via a value matrix. Standard attention heads learn a rigid mapping between search and retrieval. In this work, we first highlight how this static nature of the pairing can potentially: (a) lead to learning of redundant parameters in certain tasks, and (b) hinder generalization. To alleviate this problem, we propose a novel attention mechanism, called Compositional Attention, that replaces the standard head structure. The proposed mechanism disentangles search and retrieval and composes them in a dynamic, flexible and context-dependent manner. Through a series of numerical experiments, we show that it outperforms standard multi-head attention on a variety of tasks, including some out-of-distribution settings. Through our qualitative analysis, we demonstrate that Compositional Attention leads to dynamic specialization based on the type of retrieval needed. Our proposed mechanism generalizes multi-head attention, allows independent scaling of search and retrieval and is easy to implement in a variety of established network architectures.

2022-01-01

International Conference on Learning Representations (published)

openreview.net

Goal-driven optimization of single-neuron properties in artiﬁcial networks reveals regularization role of neural diversity and adaptation in the brain

Victor Geadah

Stefan Horoi

Giancarlo Kerg

Neurons in the brain have rich and adaptive input-output properties. Features such as diverse f-I curves and spike frequency adaptation are … (see more)known to place single neurons in optimal coding regimes when facing changing stimuli. Yet, it is still unclear how brain circuits exploit single neuron ﬂexibility, and how network-level requirements may have shaped such cellular function. To answer this question, a multi-scaled approach is needed where the computations of single neurons and of neural circuits must be considered as a complete system. In this work, we use artiﬁcial neural networks to systematically investigate single neuron input-output adaptive mechanisms, optimized in an end-to-end fashion. Throughout the optimization process, each neuron has the liberty to modify its nonlinear activation function, parametrized to mimic f-I curves of biological neurons, and to learn adaptation strategies to modify activation functions in real-time during a task. We ﬁnd that such networks show much-improved robustness to noise and changes in input statistics. Importantly, we ﬁnd that this procedure recovers precise coding strategies found in biological neurons, such as gain scaling and fractional order differentiation/integration. Using tools from dynamical systems theory, we analyze the role of these emergent single neuron properties and argue that neural diversity and adaptation plays an active regularization role that enables neural circuits to optimally propagate information across time.

2022-01-01

(published)

www.semanticscholar.org

Goal-driven optimization of single-neuron properties in artiﬁcial networks reveals regularization role of neural diversity and adaptation in the brain

Victor Geadah

Stefan Horoi

Giancarlo Kerg

2022-01-01

(published)

www.semanticscholar.org

Lazy vs hasty: linearization in deep networks impacts learning schedule based on example difficulty

Thomas George

Aristide Baratin

Among attempts at giving a theoretical account of the success of deep neural networks, a recent line of work has identified a so-called `laz… (see more)y' training regime in which the network can be well approximated by its linearization around initialization. Here we investigate the comparative effect of the lazy (linear) and feature learning (non-linear) regimes on subgroups of examples based on their difficulty. Specifically, we show that easier examples are given more weight in feature learning mode, resulting in faster training compared to more difficult ones. In other words, the non-linear dynamics tends to sequentialize the learning of examples of increasing difficulty. We illustrate this phenomenon across different ways to quantify example difficulty, including c-score, label noise, and in the presence of easy-to-learn spurious correlations. Our results reveal a new understanding of how deep networks prioritize resources across example difficulty.

2022-01-01

Trans. Mach. Learn. Res. (published)

openreview.net

Learning function from structure in neuromorphic networks

Laura E. Suárez

Bratislav Mišić

2021-08-09

Nature Machine Intelligence (published)

Learning Brain Dynamics With Coupled Low-Dimensional Nonlinear Oscillators and Deep Recurrent Networks

Germán Abrevaya

Guillaume Dumas

Aleksandr Y. Aravkin

Peng Zheng

Jean-Christophe Gagnon-Audet

James Kozloski

Pablo Polosecki

David Cox

Silvina Ponce Dawson

Guillermo Cecchi

Irina Rish

Many natural systems, especially biological ones, exhibit complex multivariate nonlinear dynamical behaviors that can be hard to capture by … (see more)linear autoregressive models. On the other hand, generic nonlinear models such as deep recurrent neural networks often require large amounts of training data, not always available in domains such as brain imaging; also, they often lack interpretability. Domain knowledge about the types of dynamics typically observed in such systems, such as a certain type of dynamical systems models, could complement purely data-driven techniques by providing a good prior. In this work, we consider a class of ordinary differential equation (ODE) models known as van der Pol (VDP) oscil lators and evaluate their ability to capture a low-dimensional representation of neural activity measured by different brain imaging modalities, such as calcium imaging (CaI) and fMRI, in different living organisms: larval zebrafish, rat, and human. We develop a novel and efficient approach to the nontrivial problem of parameters estimation for a network of coupled dynamical systems from multivariate data and demonstrate that the resulting VDP models are both accurate and interpretable, as VDP's coupling matrix reveals anatomically meaningful excitatory and inhibitory interactions across different brain subsystems. VDP outperforms linear autoregressive models (VAR) in terms of both the data fit accuracy and the quality of insight provided by the coupling matrices and often tends to generalize better to unseen data when predicting future brain activity, being comparable to and sometimes better than the recurrent neural networks (LSTMs). Finally, we demonstrate that our (generative) VDP model can also serve as a data-augmentation tool leading to marked improvements in predictive accuracy of recurrent neural networks. Thus, our work contributes to both basic and applied dimensions of neuroimaging: gaining scientific insights and improving brain-based predictive models, an area of potentially high practical importance in clinical diagnosis and neurotechnology.

2021-05-26

Neural Computation (published)

PNS-GAN: Conditional Generation of Peripheral Nerve Signals in the Wavelet Domain via Adversarial Networks

Olivier Tessier-Lariviere

Luke Y. Prince

Pascal Fortier-Poisson

Lorenz Wernisch

Oliver Armitage

Emil Hewage

Simulated datasets of neural recordings are a crucial tool in neural engineering for testing the ability of decoding algorithms to recover k… (see more)nown ground-truth. In this work, we introduce PNS-GAN, a generative adversarial network capable of producing realistic nerve recordings conditioned on physiological biomarkers. PNS-GAN operates in the wavelet domain to preserve both the timing and frequency of neural events with high resolution. PNS-GAN generates sequences of scaleograms from noise using a recurrent neural network and 2D transposed convolution layers. PNS-GAN discriminates over stacks of scaleograms with a network of 3D convolution layers. We find that our generated signal reproduces a number of characteristics of the real signal, including similarity in a canonical time-series feature-space, and contains physiologically related neural events including respiration modulation and similar distributions of afferent and efferent signalling.

2021-05-04

International IEEE/EMBS Conference on Neural Engineering (published)

Embedding Signals on Knowledge Graphs with Unbalanced Diffusion Earth Mover's Distance

Alexander Tong

Guillaume Huguet

Dennis Shung

Amine Natik

Manik Kuchroo

Smita Krishnaswamy

In modern relational machine learning it is common to encounter large graphs that arise via interactions or similarities between observation… (see more)s in many domains. Further

2021-01-01

arXiv.org (preprint)

dblp.uni-trier.de

Embedding Signals on Knowledge Graphs with Unbalanced Diffusion Earth Mover's Distance

Alexander Tong

Guillaume Huguet

Dennis L. Shung

Amine Natik

Manik Kuchroo