Guillaume Lajoie

Biographie

Guillaume Lajoie est professeur agrégé au Département de mathématiques et de statistiques (DMS) de l'Université de Montréal et membre académique principal de Mila – Institut québécois d’intelligence artificielle. Il est titulaire d'une chaire CIFAR (CCAI Canada) ainsi que d'une chaire de recherche du Canada (CRC) en calcul et interfaçage neuronaux.

Ses recherches sont positionnées à l'intersection de l'IA et des neurosciences où il développe des outils pour mieux comprendre les mécanismes d'intelligence communs aux systèmes biologiques et artificiels. Les contributions de son groupe de recherche vont des progrès des paradigmes d'apprentissage à plusieurs échelles pour les grands systèmes artificiels aux applications en neurotechnologie. Dr. Lajoie participe activement aux efforts de développement responsables de l'IA, cherchant à identifier les lignes directrices et les meilleures pratiques pour l'utilisation de l'IA dans la recherche et au-delà.

Étudiants actuels

Federico Arangath Joseph

Collaborateur·rice de recherche - ETH Zurich

Stefan Bauer

Visiteur de recherche indépendant

Superviseur⋅e principal⋅e :

Yoshua Bengio

Sangnie Bhardwaj

Doctorat - UdeM

Co-superviseur⋅e :

Hugo Larochelle

Colin Bredenberg

Postdoctorat - UdeM

Co-superviseur⋅e :

Leo Choiniere

Doctorat - UdeM

Olivier Codol

Postdoctorat - UdeM

Co-superviseur⋅e :

Doctorat - UdeM

Superviseur⋅e principal⋅e :

Doctorat - UdeM

Superviseur⋅e principal⋅e :

Leo Gagnon

Doctorat - UdeM

Juan Guerra

Maîtrise recherche - Polytechnique

Superviseur⋅e principal⋅e :

Marco Bonizzato

Site web

Nanda Harishankar Krishna

Doctorat - UdeM

Collaborateur·rice de recherche - Western Washington University (faculty; assistant prof))

Superviseur⋅e principal⋅e :

Maîtrise recherche - UdeM

Co-superviseur⋅e :

Dhanya Sridhar

tejaskasetty@gmail.com

Collaborateur·rice de recherche - UdeM

Ximeng Mao

Doctorat - UdeM

Co-superviseur⋅e :

Joelle Pineau

Abdel Mfougouon Njupoun

Doctorat - UdeM

Co-superviseur⋅e :

Doctorat - UdeM

Co-superviseur⋅e :

Amine Natik

Doctorat - UdeM

Co-superviseur⋅e :

Guy Wolf

François Paugam

Doctorat - UdeM

Superviseur⋅e principal⋅e :

Lune Bellec

Alexandre Payeur

Collaborateur·rice de recherche - UdeM

Mohammad Pezeshki

Collaborateur·rice de recherche

Superviseur⋅e principal⋅e :

Postdoctorat - McGill

Superviseur⋅e principal⋅e :

Param Raval

Collaborateur·rice alumni - UdeM

Avery Ryoo

Maîtrise recherche - UdeM

Superviseur⋅e principal⋅e :

Doctorat - UdeM

Co-superviseur⋅e :

Lune Bellec

Laura Suarez

Doctorat - McGill

Ryan Vogt

Postdoctorat - UdeM

Que nous apprennent les distributions des coefficients synaptiques au sujet de l’apprentissage dans le cerveau ?

Vivian White

Stagiaire de recherche - Western Washington University

Co-superviseur⋅e :

Doctorat - UdeM

Billets de blogue

13 juin 2024

par

Roman Pogodin

Jonathan Cornford

Arna Ghosh

Gauthier Gidel

Guillaume Lajoie

Blake Richards

Lire l'article

Publications

Sources of richness and ineffability for phenomenally conscious states

Xu Ji

Eric Elmoznino

George Deane

Axel Constant

Guillaume Dumas

Jonathan Simon

Yoshua Bengio

Abstract Conscious states—state that there is something it is like to be in—seem both rich or full of detail and ineffable or hard to fu… (voir plus)lly describe or recall. The problem of ineffability, in particular, is a longstanding issue in philosophy that partly motivates the explanatory gap: the belief that consciousness cannot be reduced to underlying physical processes. Here, we provide an information theoretic dynamical systems perspective on the richness and ineffability of consciousness. In our framework, the richness of conscious experience corresponds to the amount of information in a conscious state and ineffability corresponds to the amount of information lost at different stages of processing. We describe how attractor dynamics in working memory would induce impoverished recollections of our original experiences, how the discrete symbolic nature of language is insufficient for describing the rich and high-dimensional structure of experiences, and how similarity in the cognitive function of two individuals relates to improved communicability of their experiences to each other. While our model may not settle all questions relating to the explanatory gap, it makes progress toward a fully physicalist explanation of the richness and ineffability of conscious experience—two important aspects that seem to be part of what makes qualitative character so puzzling.

2023-02-13

ArXiv (prépublication)

arxiv.org

How gradient estimator variance and bias impact learning in neural networks

Arna Ghosh

Yuhan Helena Liu

Konrad Paul Kording

There is growing interest in understanding how real brains may approximate gradients and how gradients can be used to train neuromorphic chi… (voir plus)ps. However, neither real brains nor neuromorphic chips can perfectly follow the loss gradient, so parameter updates would necessarily use gradient estimators that have some variance and/or bias. Therefore, there is a need to understand better how variance and bias in gradient estimators impact learning dependent on network and task properties. Here, we show that variance and bias can impair learning on the training data, but some degree of variance and bias in a gradient estimator can be beneficial for generalization. We find that the ideal amount of variance and bias in a gradient estimator are dependent on several properties of the network and task: the size and activity sparsity of the network, the norm of the gradient, and the curvature of the loss landscape. As such, whether considering biologically-plausible learning algorithms or algorithms for training neuromorphic chips, researchers can analyze these properties to determine whether their approximation to gradient descent will be effective for learning given their network and task properties.

2023-02-01

ICLR.cc/2023/Conference (poster)

Reliability of CKA as a Similarity Measure in Deep Learning

MohammadReza Davari

Stefan Horoi

Amine Natik

Guy Wolf

Eugene Belilovsky

Comparing learned neural representations in neural networks is a challenging but important problem, which has been approached in different w… (voir plus)ays. The Centered Kernel Alignment (CKA) similarity metric, particularly its linear variant, has recently become a popular approach and has been widely used to compare representations of a network's different layers, of architecturally similar networks trained differently, or of models with different architectures trained on the same data. A wide variety of claims about similarity and dissimilarity of these various representations have been made using CKA results. In this work we present analysis that formally characterizes CKA sensitivity to a large class of simple transformations, which can naturally occur in the context of modern machine learning. This provides a concrete explanation to CKA sensitivity to outliers, which has been observed in past works, and to transformations that preserve the linear separability of the data, an important generalization attribute. We empirically investigate several weaknesses of the CKA similarity metric, demonstrating situations in which it gives unexpected or counterintuitive results. Finally we study approaches for modifying representations to maintain functional behaviour while changing the CKA value. Our results illustrate that, in many cases, the CKA value can be easily manipulated without substantial changes to the functional behaviour of the models, and call for caution when leveraging activation alignment metrics.

2023-02-01

ICLR.cc/2023/Conference (poster)

« Que notre cerveau soit constitué de neurones n’est pas un accident »

Roman Ikonicoff

2023-01-02

Pour la science (publié)

Formalizing locality for normative synaptic plasticity models

Colin Bredenberg

Ezekiel Williams

Cristina Savin

H OW GRADIENT ESTIMATOR VARIANCE AND BIAS COULD IMPACT LEARNING IN NEURAL CIRCUITS

Arna Ghosh

Yuhan Helena Liu

Konrad K¨ording

2023-01-01

(publié)

www.semanticscholar.org

LEAD: Min-Max Optimization from a Physical Perspective

Reyhane Askari Hemmat

Amartya Mitra

Ioannis Mitliagkas

Adversarial formulations have rekindled interest in two-player min-max games. A central obstacle in the optimization of such games is the ro… (voir plus)tational dynamics that hinder their convergence. In this paper, we show that game optimization shares dynamic properties with particle systems subject to multiple forces, and one can leverage tools from physics to improve optimization dynamics. Inspired by the physical framework, we propose LEAD, an optimizer for min-max games. Next, using Lyapunov stability theory from dynamical systems as well as spectral analysis, we study LEAD’s convergence properties in continuous and discrete time settings for a class of quadratic min-max games to demonstrate linear convergence to the Nash equilibrium. Finally, we empirically evaluate our method on synthetic setups and CIFAR-10 image generation to demonstrate improvements in GAN training.

2023-01-01

Trans. Mach. Learn. Res. (publié)

NEURAL MANIFOLDS AND GRADIENT-BASED ADAPTATION IN NEURAL-INTERFACE TASKS

Alexandre Payeur

Amy L. Orsborn

. Neural activity tends to reside on manifolds whose dimension is much lower than the dimension of the whole neural state space. Experiments… (voir plus) using brain-computer interfaces with microelectrode arrays implanted in the motor cortex of nonhuman primates tested the hypothesis that external perturbations should produce different adaptation strategies depending on how “aligned” the perturbation is with respect to a pre-existing intrinsic manifold. On the one hand, perturbations within the manifold (WM) evoked fast reassociations of existing patterns for rapid adaptation. On the other hand, perturbations outside the manifold (OM) triggered the slow emergence of new neural patterns underlying a much slower—and, without adequate training protocols, inconsistent or virtually impossible—adaptation. This suggests that the time scale and the overall difficulty of the brain to adapt depend fundamentally on the structure of neural activity. Here, we used a simplified static Gaussian model to show that gradient-descent learning could explain the differences between adaptation to WM and OM perturbations. For small learning rates, we found that the adaptation speeds were different but the model eventually adapted to both perturbations. Moreover, sufficiently large learning rates could entirely prohibit adaptation to OM perturbations while preserving adaptation to WM perturbations, in agreement with experiments. Adopting an incremental training protocol, as has been done in experiments, permitted a swift recovery of a full adaptation in the cases where OM perturbations were previously impossible to relearn. Finally, we also found that gradient descent was compatible with the reassociation mechanism on short adaptation time scales. Since gradient descent has many biologically plausible variants, our findings thus establish gradient-based learning as a plausible mechanism for adaptation under network-level constraints, with a central role for the learning rate.

NEURAL MANIFOLDS AND GRADIENT-BASED ADAPTATION IN NEURAL-INTERFACE TASKS

Alexandre Payeur

Amy L. Orsborn

2023-01-01

(published)

www.semanticscholar.org

Author Correction: Gradient-based learning drives robust representations in recurrent neural networks by balancing compression and expansion

Maxwell J. Farrell

Stefano Recanatesi

Timothy Moore

Eric Todd SheaBrown

2022-10-19

Nature Machine Intelligence (publié)

Rapidly Inferring Personalized Neurostimulation Parameters with Meta-Learning: A Case Study of Individualized Fiber Recruitment in Vagus Nerve Stimulation

Ximeng Mao

Yao-Chuan Chang

Stavros Zanos

2022-09-08

bioRxiv (prépublication)

Learning Shared Neural Manifolds from Multi-Subject FMRI Data

Jessie Huang

Je-chun Huang

Erica Lindsey Busch

Tom Wallenstein

Michal Gerasimiuk

Andrew Benz

Guy Wolf

Nicholas Turk-Browne

Smita Krishnaswamy

Functional magnetic resonance imaging (fMRI) data is collected in millions of noisy, redundant dimensions. To understand how different brain… (voir plus)s process the same stimulus, we aim to denoise the fMRI signal via a meaningful embedding space that captures the data's intrinsic structure as shared across brains. We assume that stimulus-driven responses share latent features common across subjects that are jointly discoverable. Previous approaches to this problem have relied on linear methods like principal component analysis and shared response modeling. We propose a neural network called MRMD-AE (manifold-regularized multiple- decoder, autoencoder) that learns a common embedding from multi-subject fMRI data while retaining the ability to decode individual responses. Our latent common space represents an extensible manifold (where untrained data can be mapped) and improves classification accuracy of stimulus features of unseen timepoints, as well as cross-subject translation of fMRI signals.

2022-08-22

2022 IEEE 32nd International Workshop on Machine Learning for Signal Processing (MLSP) (publié)