Guillaume Lajoie

alexandre.payeur@mila.quebec

Biographie

Guillaume Lajoie est professeur adjoint au Département de mathématiques et de statistique (DMS) de l'Université de Montréal et membre académique principal de Mila – Institut québécois d’intelligence artificielle. Il est également chercheur boursier du Fonds de recherche du Québec - Santé (FRQS). Auparavant, il a été chercheur postdoctoral à l'Institut de dynamique et d'auto-organisation Max-Planck et à l'Institut de neuro-ingénierie de l'Université de Washington. Il a obtenu son doctorat à l'Université de Washington (Seattle), au Département de mathématiques appliquées.

Ses recherches, à l’intersection de l’IA et des neurosciences, se penchent sur des questions liées aux dynamiques et aux calculs des réseaux neuronaux, avec certaines applications à la neuro-ingénierie. Ses travaux récents comprennent le développement de biais inductifs pour une meilleure propagation de l’information dans les réseaux récurrents, ainsi que le développement d'algorithmes permettant d’optimiser les interfaces cerveau-machine bidirectionnelles.

Étudiants actuels

Alexandre Payeur

Collaborateur·rice de recherche - Université de Montréal

Amine Natik

Doctorat - Université de Montréal

Co-superviseur⋅e :

natikami@mila.quebec

Colin Bredenberg

Postdoctorat - Université de Montréal

Co-superviseur⋅e :

colin.bredenberg@mila.quebec

Eric Elmoznino

Doctorat - Université de Montréal

Superviseur⋅e principal⋅e :

eric.elmoznino@mila.quebec

Doctorat - Université de Montréal

ezekiel.williams@mila.quebec

Doctorat - Université de Montréal

paugamfr@mila.quebec

hee-woon.ryoo@mila.quebec

Avery Ryoo

Maîtrise recherche - Université de Montréal

Superviseur⋅e principal⋅e :

Matt Perich

Site web

yuhan-helena.liu@mila.quebec

Helena Yuhan Liu

Collaborateur·rice de recherche - Université de Montréal

Jean-pierre Falet

Doctorat - Université de Montréal

Superviseur⋅e principal⋅e :

jean-pierre.falet@mila.quebec

Site web

Juan Guerra

Maîtrise recherche - Polytechnique Montréal

Superviseur⋅e principal⋅e :

Marco Bonizzato

juan.guerra@mila.quebec

leo.choiniere@mila.quebec

Laura Suarez

Doctorat - McGill University

suarezul@mila.quebec

Leo Choiniere

Doctorat - Université de Montréal

Leo Gagnon

Doctorat - Université de Montréal

leo.gagnon@mila.quebec

nanda.harishankar-krishna@mila.quebec

Nanda Harishankar Krishna

Doctorat - Université de Montréal

Postdoctorat - Université de Montréal

Co-superviseur⋅e :

Matt Perich

olivier.codol@mila.quebec

Pravish Sainath

Doctorat - Université de Montréal

Co-superviseur⋅e :

Pierre (Louis) Bellec

sainathp@mila.quebec

Roman Pogodin

Postdoctorat - McGill University

Superviseur⋅e principal⋅e :

Roman.Pogodin@mila.Quebec

Maîtrise recherche - Polytechnique Montréal

Superviseur⋅e principal⋅e :

Doctorat - Université de Montréal

Co-superviseur⋅e :

Hugo Larochelle

sangnie.bhardwaj@mila.quebec

Sarthak Mittal

Doctorat - Université de Montréal

Co-superviseur⋅e :

Visiteur de recherche indépendant

Superviseur⋅e principal⋅e :

stefan.bauer@mila.quebec

tejas.kasetty@mila.quebec

Tejas Kasetty

Maîtrise professionnelle - Université de Montréal

Collaborateur·rice de recherche - Polytechnique Montréal

Superviseur⋅e principal⋅e :

Marco Bonizzato

thomas.garbay@mila.quebec

Vivian White

Stagiaire de recherche - Western Washington University

Co-superviseur⋅e :

vivian.white@mila.quebec

Ximeng Mao

Doctorat - Université de Montréal

Co-superviseur⋅e :

Joelle Pineau

ximeng.mao@mila.quebec

Que nous apprennent les distributions des coefficients synaptiques au sujet de l’apprentissage dans le cerveau ?

Billets de blogue

13 juin 2024

par

Roman Pogodin

Jonathan Cornford

Arna Ghosh

Gauthier Gidel

Guillaume Lajoie

Blake Richards

Lire l'article

Publications

A connectomics-based taxonomy of mammals

Laura E. Suárez

Yossi Yovel

M. P. van den Heuvel

Olaf Sporns

Yaniv Assaf

Bratislav Mišić

Mammalian taxonomies are conventionally defined by morphological traits and genetics. How species differ in terms of neural circuits and whe… (voir plus)ther inter-species differences in neural circuit organization conform to these taxonomies is unknown. The main obstacle for the comparison of neural architectures have been differences in network reconstruction techniques, yielding species-specific connectomes that are not directly comparable to one another. Here we comprehensively chart connectome organization across the mammalian phylogenetic spectrum using a common reconstruction protocol. We analyze the mammalian MRI (MaMI) data set, a database that encompasses high-resolution ex vivo structural and diffusion magnetic resonance imaging (MRI) scans of 124 species across 12 taxonomic orders and 5 superorders, collected using a single protocol on a single scanner. We assess similarity between species connectomes using two methods: similarity of Laplacian eigenspectra and similarity of multiscale topological features. We find greater inter-species similarities among species within the same taxonomic order, suggesting the connectome organization recapitulates traditional taxonomies defined by morphology and genetics. While all connectomes retain hallmark global features and relative proportions of connection classes, inter-species variation is driven by local regional connectivity profiles. By encoding connectomes into a common frame of reference, these findings establish a foundation for investigating how neural circuits change over phylogeny, forging a link from genes to circuits to behaviour.

2022-03-12

bioRxiv (preprint)

Beyond accuracy: generalization properties of bio-plausible temporal credit assignment rules

Yuhan Helena Liu

Arna Ghosh

Eric Todd SheaBrown

openreview.net

Compositional Attention: Disentangling Search and Retrieval

Sarthak Mittal

Sharath Chandra Raparthy

Irina Rish

Multi-head, key-value attention is the backbone of transformer-like model architectures which have proven to be widely successful in recent … (voir plus)years. This attention mechanism uses multiple parallel key-value attention blocks (called heads), each performing two fundamental computations: (1) search - selection of a relevant entity from a set via query-key interaction, and (2) retrieval - extraction of relevant features from the selected entity via a value matrix. Standard attention heads learn a rigid mapping between search and retrieval. In this work, we first highlight how this static nature of the pairing can potentially: (a) lead to learning of redundant parameters in certain tasks, and (b) hinder generalization. To alleviate this problem, we propose a novel attention mechanism, called Compositional Attention, that replaces the standard head structure. The proposed mechanism disentangles search and retrieval and composes them in a dynamic, flexible and context-dependent manner. Through a series of numerical experiments, we show that it outperforms standard multi-head attention on a variety of tasks, including some out-of-distribution settings. Through our qualitative analysis, we demonstrate that Compositional Attention leads to dynamic specialization based on the type of retrieval needed. Our proposed mechanism generalizes multi-head attention, allows independent scaling of search and retrieval and is easy to implement in a variety of established network architectures.

2022-01-01

International Conference on Learning Representations (publié)

openreview.net

Goal-driven optimization of single-neuron properties in artiﬁcial networks reveals regularization role of neural diversity and adaptation in the brain

Victor Geadah

Stefan Horoi

Giancarlo Kerg

Neurons in the brain have rich and adaptive input-output properties. Features such as diverse f-I curves and spike frequency adaptation are … (voir plus)known to place single neurons in optimal coding regimes when facing changing stimuli. Yet, it is still unclear how brain circuits exploit single neuron ﬂexibility, and how network-level requirements may have shaped such cellular function. To answer this question, a multi-scaled approach is needed where the computations of single neurons and of neural circuits must be considered as a complete system. In this work, we use artiﬁcial neural networks to systematically investigate single neuron input-output adaptive mechanisms, optimized in an end-to-end fashion. Throughout the optimization process, each neuron has the liberty to modify its nonlinear activation function, parametrized to mimic f-I curves of biological neurons, and to learn adaptation strategies to modify activation functions in real-time during a task. We ﬁnd that such networks show much-improved robustness to noise and changes in input statistics. Importantly, we ﬁnd that this procedure recovers precise coding strategies found in biological neurons, such as gain scaling and fractional order differentiation/integration. Using tools from dynamical systems theory, we analyze the role of these emergent single neuron properties and argue that neural diversity and adaptation plays an active regularization role that enables neural circuits to optimally propagate information across time.

2022-01-01

(published)

www.semanticscholar.org

Goal-driven optimization of single-neuron properties in artiﬁcial networks reveals regularization role of neural diversity and adaptation in the brain

Victor Geadah

Stefan Horoi

Giancarlo Kerg

2022-01-01

(publié)

www.semanticscholar.org

Lazy vs hasty: linearization in deep networks impacts learning schedule based on example difficulty

Thomas George

Aristide Baratin

Among attempts at giving a theoretical account of the success of deep neural networks, a recent line of work has identified a so-called `laz… (voir plus)y' training regime in which the network can be well approximated by its linearization around initialization. Here we investigate the comparative effect of the lazy (linear) and feature learning (non-linear) regimes on subgroups of examples based on their difficulty. Specifically, we show that easier examples are given more weight in feature learning mode, resulting in faster training compared to more difficult ones. In other words, the non-linear dynamics tends to sequentialize the learning of examples of increasing difficulty. We illustrate this phenomenon across different ways to quantify example difficulty, including c-score, label noise, and in the presence of easy-to-learn spurious correlations. Our results reveal a new understanding of how deep networks prioritize resources across example difficulty.

2022-01-01

Trans. Mach. Learn. Res. (publié)

openreview.net

Learning function from structure in neuromorphic networks

Laura E. Suárez

Bratislav Mišić

2021-08-09

Nature Machine Intelligence (publié)

Learning Brain Dynamics With Coupled Low-Dimensional Nonlinear Oscillators and Deep Recurrent Networks

Germán Abrevaya

Guillaume Dumas

Aleksandr Y. Aravkin

Peng Zheng

Jean-Christophe Gagnon-Audet

James Kozloski

Pablo Polosecki

David Cox

Silvina Ponce Dawson

Guillermo Cecchi

Irina Rish

Many natural systems, especially biological ones, exhibit complex multivariate nonlinear dynamical behaviors that can be hard to capture by … (voir plus)linear autoregressive models. On the other hand, generic nonlinear models such as deep recurrent neural networks often require large amounts of training data, not always available in domains such as brain imaging; also, they often lack interpretability. Domain knowledge about the types of dynamics typically observed in such systems, such as a certain type of dynamical systems models, could complement purely data-driven techniques by providing a good prior. In this work, we consider a class of ordinary differential equation (ODE) models known as van der Pol (VDP) oscil lators and evaluate their ability to capture a low-dimensional representation of neural activity measured by different brain imaging modalities, such as calcium imaging (CaI) and fMRI, in different living organisms: larval zebrafish, rat, and human. We develop a novel and efficient approach to the nontrivial problem of parameters estimation for a network of coupled dynamical systems from multivariate data and demonstrate that the resulting VDP models are both accurate and interpretable, as VDP's coupling matrix reveals anatomically meaningful excitatory and inhibitory interactions across different brain subsystems. VDP outperforms linear autoregressive models (VAR) in terms of both the data fit accuracy and the quality of insight provided by the coupling matrices and often tends to generalize better to unseen data when predicting future brain activity, being comparable to and sometimes better than the recurrent neural networks (LSTMs). Finally, we demonstrate that our (generative) VDP model can also serve as a data-augmentation tool leading to marked improvements in predictive accuracy of recurrent neural networks. Thus, our work contributes to both basic and applied dimensions of neuroimaging: gaining scientific insights and improving brain-based predictive models, an area of potentially high practical importance in clinical diagnosis and neurotechnology.

2021-05-26

Neural Computation (publié)

PNS-GAN: Conditional Generation of Peripheral Nerve Signals in the Wavelet Domain via Adversarial Networks

Olivier Tessier-Lariviere

Luke Y. Prince

Pascal Fortier-Poisson

Lorenz Wernisch

Oliver Armitage

Emil Hewage

Simulated datasets of neural recordings are a crucial tool in neural engineering for testing the ability of decoding algorithms to recover k… (voir plus)nown ground-truth. In this work, we introduce PNS-GAN, a generative adversarial network capable of producing realistic nerve recordings conditioned on physiological biomarkers. PNS-GAN operates in the wavelet domain to preserve both the timing and frequency of neural events with high resolution. PNS-GAN generates sequences of scaleograms from noise using a recurrent neural network and 2D transposed convolution layers. PNS-GAN discriminates over stacks of scaleograms with a network of 3D convolution layers. We find that our generated signal reproduces a number of characteristics of the real signal, including similarity in a canonical time-series feature-space, and contains physiologically related neural events including respiration modulation and similar distributions of afferent and efferent signalling.

2021-05-04

International IEEE/EMBS Conference on Neural Engineering (publié)

Embedding Signals on Knowledge Graphs with Unbalanced Diffusion Earth Mover's Distance

Alexander Tong

Guillaume Huguet

Dennis Shung

Amine Natik

Manik Kuchroo

Smita Krishnaswamy

In modern relational machine learning it is common to encounter large graphs that arise via interactions or similarities between observation… (voir plus)s in many domains. Further

2021-01-01

arXiv.org (prépublication)

dblp.uni-trier.de

Embedding Signals on Knowledge Graphs with Unbalanced Diffusion Earth Mover's Distance

Alexander Tong

Guillaume Huguet

Dennis L. Shung

Amine Natik

Manik Kuchroo