Guillaume Lajoie

alexandre.payeur@mila.quebec

Biographie

Guillaume Lajoie est professeur adjoint au Département de mathématiques et de statistique (DMS) de l'Université de Montréal et membre académique principal de Mila – Institut québécois d’intelligence artificielle. Il est également chercheur boursier du Fonds de recherche du Québec - Santé (FRQS). Auparavant, il a été chercheur postdoctoral à l'Institut de dynamique et d'auto-organisation Max-Planck et à l'Institut de neuro-ingénierie de l'Université de Washington. Il a obtenu son doctorat à l'Université de Washington (Seattle), au Département de mathématiques appliquées.

Ses recherches, à l’intersection de l’IA et des neurosciences, se penchent sur des questions liées aux dynamiques et aux calculs des réseaux neuronaux, avec certaines applications à la neuro-ingénierie. Ses travaux récents comprennent le développement de biais inductifs pour une meilleure propagation de l’information dans les réseaux récurrents, ainsi que le développement d'algorithmes permettant d’optimiser les interfaces cerveau-machine bidirectionnelles.

Étudiants actuels

Alexandre Payeur

Collaborateur·rice de recherche - Université de Montréal

Amine Natik

Doctorat - Université de Montréal

Co-superviseur⋅e :

Guy Wolf

natikami@mila.quebec

Colin Bredenberg

Postdoctorat - Université de Montréal

Co-superviseur⋅e :

Blake Richards

colin.bredenberg@mila.quebec

Eric Elmoznino

Doctorat - Université de Montréal

Superviseur⋅e principal⋅e :

Yoshua Bengio

eric.elmoznino@mila.quebec

Doctorat - Université de Montréal

ezekiel.williams@mila.quebec

Doctorat - Université de Montréal

paugamfr@mila.quebec

hee-woon.ryoo@mila.quebec

Avery Ryoo

Maîtrise recherche - Université de Montréal

Superviseur⋅e principal⋅e :

Matt Perich

Site web

yuhan-helena.liu@mila.quebec

Helena Yuhan Liu

Collaborateur·rice de recherche - Université de Montréal

jean-pierre.falet@mila.quebec

Jean-pierre Falet

Doctorat - Université de Montréal

Superviseur⋅e principal⋅e :

Yoshua Bengio

Site web

Juan Guerra

Maîtrise recherche - Polytechnique Montréal

Superviseur⋅e principal⋅e :

Marco Bonizzato

juan.guerra@mila.quebec

leo.choiniere@mila.quebec

Laura Suarez

Doctorat - McGill University

suarezul@mila.quebec

Leo Choiniere

Doctorat - Université de Montréal

Leo Gagnon

Doctorat - Université de Montréal

leo.gagnon@mila.quebec

nanda.harishankar-krishna@mila.quebec

Nanda Harishankar Krishna

Doctorat - Université de Montréal

Postdoctorat - Université de Montréal

Co-superviseur⋅e :

Matt Perich

olivier.codol@mila.quebec

Pravish Sainath

Doctorat - Université de Montréal

Co-superviseur⋅e :

Pierre (Louis) Bellec

sainathp@mila.quebec

Roman.Pogodin@mila.Quebec

Roman Pogodin

Postdoctorat - McGill University

Superviseur⋅e principal⋅e :

Blake Richards

Maîtrise recherche - Polytechnique Montréal

Superviseur⋅e principal⋅e :

Doctorat - Université de Montréal

Co-superviseur⋅e :

Hugo Larochelle

sangnie.bhardwaj@mila.quebec

Sarthak Mittal

Doctorat - Université de Montréal

Co-superviseur⋅e :

Visiteur de recherche indépendant

Superviseur⋅e principal⋅e :

Yoshua Bengio

stefan.bauer@mila.quebec

tejas.kasetty@mila.quebec

Tejas Kasetty

Maîtrise professionnelle - Université de Montréal

Collaborateur·rice de recherche - Polytechnique Montréal

Superviseur⋅e principal⋅e :

Marco Bonizzato

thomas.garbay@mila.quebec

Vivian White

Stagiaire de recherche - Western Washington University

Co-superviseur⋅e :

Guy Wolf

vivian.white@mila.quebec

Ximeng Mao

Doctorat - Université de Montréal

Co-superviseur⋅e :

Joelle Pineau

ximeng.mao@mila.quebec

Que nous apprennent les distributions des coefficients synaptiques au sujet de l’apprentissage dans le cerveau ?

Billets de blogue

13 juin 2024

par

Roman Pogodin

Jonathan Cornford

Arna Ghosh

Gauthier Gidel

Guillaume Lajoie

Blake Richards

Lire l'article

Publications

NEURAL MANIFOLDS AND GRADIENT-BASED ADAPTATION IN NEURAL-INTERFACE TASKS

Alexandre Payeur

Amy L. Orsborn

. Neural activity tends to reside on manifolds whose dimension is much lower than the dimension of the whole neural state space. Experiments… (voir plus) using brain-computer interfaces with microelectrode arrays implanted in the motor cortex of nonhuman primates tested the hypothesis that external perturbations should produce different adaptation strategies depending on how “aligned” the perturbation is with respect to a pre-existing intrinsic manifold. On the one hand, perturbations within the manifold (WM) evoked fast reassociations of existing patterns for rapid adaptation. On the other hand, perturbations outside the manifold (OM) triggered the slow emergence of new neural patterns underlying a much slower—and, without adequate training protocols, inconsistent or virtually impossible—adaptation. This suggests that the time scale and the overall difficulty of the brain to adapt depend fundamentally on the structure of neural activity. Here, we used a simplified static Gaussian model to show that gradient-descent learning could explain the differences between adaptation to WM and OM perturbations. For small learning rates, we found that the adaptation speeds were different but the model eventually adapted to both perturbations. Moreover, sufficiently large learning rates could entirely prohibit adaptation to OM perturbations while preserving adaptation to WM perturbations, in agreement with experiments. Adopting an incremental training protocol, as has been done in experiments, permitted a swift recovery of a full adaptation in the cases where OM perturbations were previously impossible to relearn. Finally, we also found that gradient descent was compatible with the reassociation mechanism on short adaptation time scales. Since gradient descent has many biologically plausible variants, our findings thus establish gradient-based learning as a plausible mechanism for adaptation under network-level constraints, with a central role for the learning rate.

2023-01-01

(publié)

www.semanticscholar.org

NEURAL MANIFOLDS AND GRADIENT-BASED ADAPTATION IN NEURAL-INTERFACE TASKS

Alexandre Payeur

Amy L. Orsborn

2023-01-01

(published)

www.semanticscholar.org

Author Correction: Gradient-based learning drives robust representations in recurrent neural networks by balancing compression and expansion

Maxwell J. Farrell

Stefano Recanatesi

Timothy Moore

Eric Todd SheaBrown

2022-10-19

Nature Machine Intelligence (publié)

Rapidly Inferring Personalized Neurostimulation Parameters with Meta-Learning: A Case Study of Individualized Fiber Recruitment in Vagus Nerve Stimulation

Ximeng Mao

Yao-Chuan Chang

Stavros Zanos

2022-09-08

bioRxiv (prépublication)

Learning Shared Neural Manifolds from Multi-Subject FMRI Data

Jessie Huang

Je-chun Huang

Erica Lindsey Busch

Tom Wallenstein

Michal Gerasimiuk

Andrew Benz

Guy Wolf

Nicholas Turk-Browne

Smita Krishnaswamy

Functional magnetic resonance imaging (fMRI) data is collected in millions of noisy, redundant dimensions. To understand how different brain… (voir plus)s process the same stimulus, we aim to denoise the fMRI signal via a meaningful embedding space that captures the data's intrinsic structure as shared across brains. We assume that stimulus-driven responses share latent features common across subjects that are jointly discoverable. Previous approaches to this problem have relied on linear methods like principal component analysis and shared response modeling. We propose a neural network called MRMD-AE (manifold-regularized multiple- decoder, autoencoder) that learns a common embedding from multi-subject fMRI data while retaining the ability to decode individual responses. Our latent common space represents an extensible manifold (where untrained data can be mapped) and improves classification accuracy of stimulus features of unseen timepoints, as well as cross-subject translation of fMRI signals.

2022-08-22

2022 IEEE 32nd International Workshop on Machine Learning for Signal Processing (MLSP) (publié)

openreview.net

Dynamic compression and expansion in a classifying recurrent network

Matthew Farrell

Maxwell J. Farrell

Stefano Recanatesi

Timothy Moore

Eric Todd SheaBrown

Recordings of neural circuits in the brain reveal extraordinary dynamical richness and high variability. At the same time, dimensionality re… (voir plus)duction techniques generally uncover low-dimensional structures underlying these dynamics when tasks are performed. In general, it is still an open question what determines the dimensionality of activity in neural circuits, and what the functional role of this dimensionality in task learning is. In this work we probe these issues using a recurrent artificial neural network (RNN) model trained by stochastic gradient descent to discriminate inputs. The RNN family of models has recently shown promise in revealing principles behind brain function. Through simulations and mathematical analysis, we show how the dimensionality of RNN activity depends on the task parameters and evolves over time and over stages of learning. We find that common solutions produced by the network naturally compress dimensionality, while variability-inducing chaos can expand it. We show how chaotic networks balance these two factors to solve the discrimination task with high accuracy and good generalization properties. These findings shed light on mechanisms by which artificial neural networks solve tasks while forming compact representations that may generalize well.

2022-06-22

Nature Machine Intelligence (publié)

On Neural Architecture Inductive Biases for Relational Tasks

Giancarlo Kerg

Sarthak Mittal

Current deep learning approaches have shown good in-distribution generalization performance, but struggle with out-of-distribution generaliz… (voir plus)ation. This is especially true in the case of tasks involving abstract relations like recognizing rules in sequences, as we find in many intelligence tests. Recent work has explored how forcing relational representations to remain distinct from sensory representations, as it seems to be the case in the brain, can help artificial systems. Building on this work, we further explore and formalize the advantages afforded by 'partitioned' representations of relations and sensory details, and how this inductive bias can help recompose learned relational structure in newly encountered settings. We introduce a simple architecture based on similarity scores which we name Compositional Relational Network (CoRelNet). Using this model, we investigate a series of inductive biases that ensure abstract relations are learned and represented distinctly from sensory data, and explore their effects on out-of-distribution generalization for a series of relational psychophysics tasks. We find that simple architectural choices can outperform existing models in out-of-distribution generalization. Together, these results show that partitioning relational representations from other information streams may be a simple way to augment existing network architectures' robustness when performing out-of-distribution relational computations.

2022-06-09

ArXiv (prépublication)

arxiv.org

Performance-gated deliberation: A context-adapted strategy in which urgency is opportunity cost

Maximilian Puelma Touzel

Paul Cisek

2022-05-01

PLoS Comput. Biol. (publié)

Performance-gated deliberation: A context-adapted strategy in which urgency is opportunity cost

Maximilian Puelma Touzel

Paul Cisek

Finding the right amount of deliberation, between insufficient and excessive, is a hard decision making problem that depends on the value we… (voir plus) place on our time. Average-reward, putatively encoded by tonic dopamine, serves in existing reinforcement learning theory as the opportunity cost of time, including deliberation time. Importantly, this cost can itself vary with the environmental context and is not trivial to estimate. Here, we propose how the opportunity cost of deliberation can be estimated adaptively on multiple timescales to account for non-stationary contextual factors. We use it in a simple decision-making heuristic based on average-reward reinforcement learning (AR-RL) that we call Performance-Gated Deliberation (PGD). We propose PGD as a strategy used by animals wherein deliberation cost is implemented directly as urgency, a previously characterized neural signal effectively controlling the speed of the decision-making process. We show PGD outperforms AR-RL solutions in explaining behaviour and urgency of non-human primates in a context-varying random walk prediction task and is consistent with relative performance and urgency in a context-varying random dot motion task. We make readily testable predictions for both neural activity and behaviour.

2022-05-01

PLoS Comput. Biol. (published)

From Points to Functions: Infinite-dimensional Representations in Diffusion Models

Sarthak Mittal

Stefan Bauer

Arash Mehrjou

Diffusion-based generative models learn to iteratively transfer unstructured noise to a complex target distribution as opposed to Generative… (voir plus) Adversarial Networks (GANs) or the decoder of Variational Autoencoders (VAEs) which produce samples from the target distribution in a single step. Thus, in diffusion models every sample is naturally connected to a random trajectory which is a solution to a learned stochastic differential equation (SDE). Generative models are only concerned with the final state of this trajectory that delivers samples from the desired distribution. Abstreiter et. al showed that these stochastic trajectories can be seen as continuous filters that wash out information along the way. Consequently, it is reasonable to ask if there is an intermediate time step at which the preserved information is optimal for a given downstream task. In this work, we show that a combination of information content from different time steps gives a strictly better representation for the downstream task. We introduce an attention and recurrence based modules that ``learn to mix'' information content of various time-steps such that the resultant representation leads to superior performance in downstream tasks.

2022-03-28

ICLR.cc/2022/Workshop/DGM4HSD (poster)

openreview.net

Inductive Biases for Relational Tasks

Giancarlo Kerg

Sarthak Mittal

Current deep learning approaches have shown good in-distribution performance but struggle in out-of-distribution settings. This is especiall… (voir plus)y true in the case of tasks involving abstract relations like recognizing rules in sequences, as required in many intelligence tests. In contrast, our brains are remarkably flexible at such tasks, an attribute that is likely linked to anatomical constraints on computations. Inspired by this, recent work has explored how enforcing that relational representations remain distinct from sensory representations can help artificial systems. Building on this work, we further explore and formalize the advantages afforded by ``partitioned'' representations of relations and sensory details. We investigate inductive biases that ensure abstract relations are learned and represented distinctly from sensory data across several neural network architectures and show that they outperform existing architectures on out-of-distribution generalization for various relational tasks. These results show that partitioning relational representations from other information streams may be a simple way to augment existing network architectures' robustness when performing relational computations.

2022-03-25

ICLR.cc/2022/Workshop/OSC (poster)

openreview.net

A connectomics-based taxonomy of mammals

Laura E. Suárez

Yossi Yovel

Martijn P. van den Heuvel

Olaf Sporns

Yaniv Assaf

Bratislav Mišić

2022-03-12

bioRxiv (prépublication)