Guillaume Lajoie

alexandre.payeur@mila.quebec

Biographie

Guillaume Lajoie est professeur adjoint au Département de mathématiques et de statistique (DMS) de l'Université de Montréal et membre académique principal de Mila – Institut québécois d’intelligence artificielle. Il est également chercheur boursier du Fonds de recherche du Québec - Santé (FRQS). Auparavant, il a été chercheur postdoctoral à l'Institut de dynamique et d'auto-organisation Max-Planck et à l'Institut de neuro-ingénierie de l'Université de Washington. Il a obtenu son doctorat à l'Université de Washington (Seattle), au Département de mathématiques appliquées.

Ses recherches, à l’intersection de l’IA et des neurosciences, se penchent sur des questions liées aux dynamiques et aux calculs des réseaux neuronaux, avec certaines applications à la neuro-ingénierie. Ses travaux récents comprennent le développement de biais inductifs pour une meilleure propagation de l’information dans les réseaux récurrents, ainsi que le développement d'algorithmes permettant d’optimiser les interfaces cerveau-machine bidirectionnelles.

Étudiants actuels

Alexandre Payeur

Collaborateur·rice de recherche - Université de Montréal

Amine Natik

Doctorat - Université de Montréal

Co-superviseur⋅e :

colin.bredenberg@mila.quebec

natikami@mila.quebec

Colin Bredenberg

Postdoctorat - Université de Montréal

Co-superviseur⋅e :

Blake Richards

Eric Elmoznino

Doctorat - Université de Montréal

Superviseur⋅e principal⋅e :

eric.elmoznino@mila.quebec

Doctorat - Université de Montréal

ezekiel.williams@mila.quebec

Doctorat - Université de Montréal

paugamfr@mila.quebec

hee-woon.ryoo@mila.quebec

Avery Ryoo

Maîtrise recherche - Université de Montréal

Superviseur⋅e principal⋅e :

Matt Perich

Site web

yuhan-helena.liu@mila.quebec

Helena Yuhan Liu

Collaborateur·rice de recherche - Université de Montréal

Jean-pierre Falet

Doctorat - Université de Montréal

Superviseur⋅e principal⋅e :

jean-pierre.falet@mila.quebec

Site web

Juan Guerra

Maîtrise recherche - Polytechnique Montréal

Superviseur⋅e principal⋅e :

Marco Bonizzato

juan.guerra@mila.quebec

leo.choiniere@mila.quebec

Laura Suarez

Doctorat - McGill University

suarezul@mila.quebec

Leo Choiniere

Doctorat - Université de Montréal

Leo Gagnon

Doctorat - Université de Montréal

leo.gagnon@mila.quebec

nanda.harishankar-krishna@mila.quebec

Nanda Harishankar Krishna

Doctorat - Université de Montréal

Postdoctorat - Université de Montréal

Co-superviseur⋅e :

Matt Perich

olivier.codol@mila.quebec

Pravish Sainath

Doctorat - Université de Montréal

Co-superviseur⋅e :

Pierre (Louis) Bellec

sainathp@mila.quebec

Roman.Pogodin@mila.Quebec

Roman Pogodin

Postdoctorat - McGill University

Superviseur⋅e principal⋅e :

Blake Richards

Maîtrise recherche - Polytechnique Montréal

Superviseur⋅e principal⋅e :

Doctorat - Université de Montréal

Co-superviseur⋅e :

Hugo Larochelle

sangnie.bhardwaj@mila.quebec

Sarthak Mittal

Doctorat - Université de Montréal

Co-superviseur⋅e :

Visiteur de recherche indépendant

Superviseur⋅e principal⋅e :

stefan.bauer@mila.quebec

tejas.kasetty@mila.quebec

Tejas Kasetty

Maîtrise professionnelle - Université de Montréal

Collaborateur·rice de recherche - Polytechnique Montréal

Superviseur⋅e principal⋅e :

Marco Bonizzato

thomas.garbay@mila.quebec

Vivian White

Stagiaire de recherche - Western Washington University

Co-superviseur⋅e :

vivian.white@mila.quebec

Ximeng Mao

Doctorat - Université de Montréal

Co-superviseur⋅e :

Joelle Pineau

ximeng.mao@mila.quebec

Que nous apprennent les distributions des coefficients synaptiques au sujet de l’apprentissage dans le cerveau ?

Billets de blogue

13 juin 2024

par

Roman Pogodin

Jonathan Cornford

Arna Ghosh

Gauthier Gidel

Guillaume Lajoie

Blake Richards

Lire l'article

Publications

Using neural biomarkers to personalize dosing of vagus nerve stimulation

Antonin Berthon

Lorenz Wernisch

Myrta Stoukidi

Michael Thornton

Olivier Tessier-Lariviere

Pascal Fortier-Poisson

Jorin Mamen

Max Pinkney

Susannah Lee

Elvijs Sarkans

Luca Annecchino

Ben Appleton

Philip Garsed

Bret Patterson

Samuel Gonshaw

Matjaž Jakopec

Sudhakaran Shunmugam

Tristan Edwards

Aleksi Tukiainen

Joel Jennings … (voir 3 de plus)

Emil Hewage

Oliver Armitage

2024-06-17

Bioelectronic Medicine (publié)

Assistive sensory-motor perturbations influence learned neural representations

Pavithra Rajeswaran

Alexandre Payeur

Amy L. Orsborn

Task errors are used to learn and refine motor skills. We investigated how task assistance influences learned neural representations using B… (voir plus)rain-Computer Interfaces (BCIs), which map neural activity into movement via a decoder. We analyzed motor cortex activity as monkeys practiced BCI with a decoder that adapted to improve or maintain performance over days. Population dimensionality remained constant or increased with learning, counter to trends with non-adaptive BCIs. Yet, over time, task information was contained in a smaller subset of neurons or population modes. Moreover, task information was ultimately stored in neural modes that occupied a small fraction of the population variance. An artificial neural network model suggests the adaptive decoders contribute to forming these compact neural representations. Our findings show that assistive decoders manipulate error information used for long-term learning computations, like credit assignment, which informs our understanding of motor learning and has implications for designing real-world BCIs.

2024-03-20

bioRxiv (prépublication)

Online Bayesian optimization of vagus nerve stimulation.

Lorenz Wernisch

Tristan Edwards

Antonin Berthon

Olivier Tessier-Lariviere

Elvijs Sarkans

Myrta Stoukidi

Pascal Fortier-Poisson

Max Pinkney

Michael Thornton

Catherine Hanley

Susannah Lee

Joel Jennings

Ben Appleton

Philip Garsed

Bret Patterson

Buttinger Will

Samuel Gonshaw

Matjaž Jakopec

Sudhakaran Shunmugam

Jorin Mamen … (voir 4 de plus)

Aleksi Tukiainen

Oliver Armitage

Emil Hewage

OBJECTIVE In bioelectronic medicine, neuromodulation therapies induce neural signals to the brain or organs, modifying their function. Stimu… (voir plus)lation devices capable of triggering exogenous neural signals using electrical waveforms require a complex and multi-dimensional parameter space to control such waveforms. Determining the best combination of parameters (waveform optimization or dosing) for treating a particular patient's illness is therefore challenging. Comprehensive parameter searching for an optimal stimulation effect is often infeasible in a clinical setting due to the size of the parameter space. Restricting this space, however, may lead to suboptimal therapeutic results, reduced responder rates, and adverse effects. Approach. As an alternative to a full parameter search, we present a flexible machine learning, data acquisition, and processing framework for optimizing neural stimulation parameters, requiring as few steps as possible using Bayesian optimization. This optimization builds a model of the neural and physiological responses to stimulations, enabling it to optimize stimulation parameters and provide estimates of the accuracy of the response model. The vagus nerve innervates, among other thoracic and visceral organs, the heart, thus controlling heart rate, making it an ideal candidate for demonstrating the effectiveness of our approach. Main results. The efficacy of our optimization approach was first evaluated on simulated neural responses, then applied to vagus nerve stimulation intraoperatively in porcine subjects. Optimization converged quickly on parameters achieving target heart rates and optimizing neural B-fiber activations despite high intersubject variability. Significance. An optimized stimulation waveform was achieved in real time with far fewer stimulations than required by alternative optimization strategies, thus minimizing exposure to side effects. Uncertainty estimates helped avoiding stimulations outside a safe range. Our approach shows that a complex set of neural stimulation parameters can be optimized in real-time for a patient to achieve a personalized precision dosing. .

2024-03-13

Journal of Neural Engineering (publié)

Learning and Aligning Structured Random Feature Networks

Vivian White

Muawiz Sajjad Chaudhary

Kameron Decker Harris

Artificial neural networks (ANNs) are considered ``black boxes'' due to the difficulty of interpreting their learned weights. While choosin… (voir plus)g the best features is not well understood, random feature networks (RFNs) and wavelet scattering ground some ANN learning mechanisms in function space with tractable mathematics. Meanwhile, the genetic code has evolved over millions of years, shaping the brain to devlop variable neural circuits with reliable structure that resemble RFNs. We explore a similar approach, embedding neuro-inspired, wavelet-like weights into multilayer RFNs. These can outperform scattering and have kernels that describe their function space at large width. We build learnable and deeper versions of these models where we can optimize separate spatial and channel covariances of the convolutional weight distributions. We find that these networks can perform comparatively with conventional ANNs while dramatically reducing the number of trainable parameters. Channel covariances are most influential, and both weight and activation alignment are needed for classification performance. Our work outlines how neuro-inspired configurations may lead to better performance in key cases and offers a potentially tractable reduced model for ANN learning.

2024-03-02

ICLR.cc/2024/Workshop/Re-Align (poster)

Learning and Aligning Structured Random Feature Networks

Vivian White

Muawiz Sajjad Chaudhary

Kameron Decker Harris

Artificial neural networks (ANNs) are considered "black boxes'' due to the difficulty of interpreting their learned weights. While choosing… (voir plus) the best features is not well understood, random feature networks (RFNs) and wavelet scattering ground some ANN learning mechanisms in function space with tractable mathematics. Meanwhile, the genetic code has evolved over millions of years, shaping the brain to develop variable neural circuits with reliable structure that resemble RFNs. We explore a similar approach, embedding neuro-inspired, wavelet-like weights into multilayer RFNs. These can outperform scattering and have kernels that describe their function space at large width. We build learnable and deeper versions of these models where we can optimize separate spatial and channel covariances of the convolutional weight distributions. We find that these networks can perform comparatively with conventional ANNs while dramatically reducing the number of trainable parameters. Channel covariances are most influential, and both weight and activation alignment are needed for classification performance. Our work outlines how neuro-inspired configurations may lead to better performance in key cases and offers a potentially tractable reduced model for ANN learning.

2024-03-02

ICLR.cc/2024/Workshop/Re-Align (poster)

Sources of richness and ineffability for phenomenally conscious states

Xu Ji

Eric Elmoznino

George Deane

Axel Constant

Guillaume Dumas

Jonathan Simon

2024-03-01

Neuroscience of Consciousness (publié)

arxiv.org

Gaussian-process-based Bayesian optimization for neurostimulation interventions in rats

Léo Choinière

Rose Guay-Hottin

Rémi Picard

Marco Bonizzato

Numa Dancause

2024-02-14

STAR Protocols (publié)

Connectome-based reservoir computing with the conn2res toolbox

Laura E. Suárez

Agoston Mihalik

Filip Milisav

Kenji Marshall

Mingze Li

Petra E. Vértes

Bratislav Mišić

2024-01-22

Nature Communications (publié)

Amortizing intractable inference in large language models

Edward J Hu

Moksh J. Jain

Eric Elmoznino

Younesse Kaddar

Nikolay Malkin

Autoregressive large language models (LLMs) compress knowledge from their training data through next-token conditional distributions. This l… (voir plus)imits tractable querying of this knowledge to start-to-end autoregressive sampling. However, many tasks of interest -- including sequence continuation, infilling, and other forms of constrained generation -- involve sampling from intractable posterior distributions. We address this limitation by using amortized Bayesian inference to sample from these intractable posteriors. Such amortization is algorithmically achieved by fine-tuning LLMs via diversity-seeking reinforcement learning algorithms: generative flow networks (GFlowNets). We empirically demonstrate that this distribution-matching paradigm of LLM fine-tuning can serve as an effective alternative to maximum-likelihood training and reward-maximizing policy optimization. As an important application, we interpret chain-of-thought reasoning as a latent variable modeling problem and demonstrate that our approach enables data-efficient adaptation of LLMs to tasks that require multi-step rationalization and tool use.

2024-01-16

ICLR.cc/2024/Conference (présentation orale)

Delta-AI: Local objectives for amortized inference in sparse graphical models

Jean-Pierre R. Falet

Hae Beom Lee

Nikolay Malkin

Chen Sun

Dragos Secrieru

Dinghuai Zhang

We present a new algorithm for amortized inference in sparse probabilistic graphical models (PGMs), which we call …

2024-01-16

ICLR.cc/2024/Conference (poster)

How connectivity structure shapes rich and lazy learning in neural circuits

Yuhan Helena Liu

Aristide Baratin

Jonathan Cornford

Stefan Mihalas

Eric Todd SheaBrown

In theoretical neuroscience, recent work leverages deep learning tools to explore how some network attributes critically influence its learn… (voir plus)ing dynamics. Notably, initial weight distributions with small (resp. large) variance may yield a rich (resp. lazy) regime, where significant (resp. minor) changes to network states and representation are observed over the course of learning. However, in biology, neural circuit connectivity generally has a low-rank structure and therefore differs markedly from the random initializations generally used for these studies. As such, here we investigate how the structure of the initial weights — in particular their effective rank — influences the network learning regime. Through both empirical and theoretical analyses, we discover that high-rank initializations typically yield smaller network changes indicative of lazier learning, a finding we also confirm with experimentally-driven initial connectivity in recurrent neural networks. Conversely, low-rank initialization biases learning towards richer learning. Importantly, however, as an exception to this rule, we find lazier learning can still occur with a low-rank initialization that aligns with task and data statistics. Our research highlights the pivotal role of initial weight structures in shaping learning regimes, with implications for metabolic costs of plasticity and risks of catastrophic forgetting.

2024-01-16

ICLR.cc/2024/Conference (poster)

Leveraging Unpaired Data for Vision-Language Generative Models via Cycle Consistency

Tianhong Li

Sangnie Bhardwaj

Yonglong Tian

Han Zhang

Jarred Barber

Dina Katabi

Huiwen Chang

Dilip Krishnan

Current vision-language generative models rely on expansive corpora of paired image-text data to attain optimal performance and generalizati… (voir plus)on capabilities. However, automatically collecting such data (e.g. via large-scale web scraping) leads to low quality and poor image-text correlation, while human annotation is more accurate but requires significant manual effort and expense. We introduce

2024-01-16

ICLR.cc/2024/Conference (spotlight)