Blake Richards

Biographie

Blake Richards est professeur agrégé à l'École d'informatique et au Département de neurologie et de neurochirurgie de l'Université McGill et membre du corps professoral de Mila – Institut québécois d’intelligence artificielle. Ses recherches se situent à l'intersection des neurosciences et de l'intelligence artificielle. Son laboratoire étudie les principes universels de l'intelligence qui s'appliquent aux agents naturels et artificiels. Il a reçu plusieurs distinctions pour ses travaux, notamment une bourse Arthur-B.-McDonald du Conseil de recherches en sciences naturelles et en génie du Canada (CRSNG) en 2022, le Prix du jeune chercheur de l'Association canadienne des neurosciences en 2019 et une chaire en IA Canada-CIFAR en 2018. M. Richards a en outre été titulaire d'une bourse postdoctorale Banting à l'hôpital SickKids de 2011 à 2013. Il a obtenu un doctorat en neurosciences de l'Université d'Oxford en 2010 et une licence en sciences cognitives et en IA de l'Université de Toronto en 2004.

Étudiants actuels

Aliya Affdal

Stagiaire de recherche - UdeM

Benjamin Alsbury-Nealy

Doctorat - McGill

benjamin@silicolabs.ca

Stella Bae

Visiteur de recherche indépendant - Seoul National University

Antoine Boudreau LeBlanc

Postdoctorat - McGill

Colin Bredenberg

Postdoctorat - UdeM

Superviseur⋅e principal⋅e :

aleksei.efremov@mail.mcgill.ca

Ethan Caballero

Doctorat - McGill

Co-superviseur⋅e :

Doctorat - McGill

Superviseur⋅e principal⋅e :

Doctorat - McGill

Postdoctorat - McGill

Alex Efremov

Doctorat - McGill

Arna Ghosh

Doctorat - McGill

Adel Halawa

Doctorat - McGill

Danny Han

Visiteur de recherche indépendant - Seoul National University

Doctorat - McGill

Angela Hu

Stagiaire de recherche - McGill

Ann Huang

Collaborateur·rice alumni

Doctorat - McGill

Visiteur de recherche indépendant - ETH Zurich

Divyansha Lachi

Collaborateur·rice de recherche - Georgia Tech

Postdoctorat - McGill

zixuan.li3@mail.mcgill.ca

Zixuan Li Li

Baccalauréat - McGill

Dongyan Lin

Doctorat - McGill

Maîtrise recherche - McGill

Toky Raharison Ralambomihanta

Abdel Mfougouon Njupoun

Doctorat - UdeM

Superviseur⋅e principal⋅e :

Baccalauréat - McGill

Krystal Pan

Maîtrise recherche - McGill

Adrien Peyrache

Visiteur de recherche indépendant

Roman Pogodin

Postdoctorat - McGill

Co-superviseur⋅e :

Baccalauréat - McGill

tokiniaina.raharisonralambomihan@mail.mcgill.ca

Alexis Roger

Doctorat - McGill

Co-superviseur⋅e :

Irina Rish

Ali Saheb Pasand

Doctorat - McGill

Co-superviseur⋅e :

Pablo Samuel Castro

Mandana Samiei

Doctorat - McGill

Superviseur⋅e principal⋅e :

Doina Precup

samiemandana@gmail.com

Aidan Sirbu

Maîtrise recherche - McGill

Co-superviseur⋅e :

Shahab Bakhtiari

Josh Tindall

Maîtrise recherche - McGill

Mashbayar Tugsbayar

Doctorat - McGill

Charlotte Volk

Maîtrise recherche - McGill

Co-superviseur⋅e :

Shahab Bakhtiari

Que nous apprennent les distributions des coefficients synaptiques au sujet de l’apprentissage dans le cerveau ?

Hee-Hwan Wang

Visiteur de recherche indépendant - Seoul National University

Maren Wehrheim

Visiteur de recherche indépendant - York University

Doctorat - McGill

Doctorat - Concordia

Superviseur⋅e principal⋅e :

Billets de blogue

13 juin 2024

par

Roman Pogodin

Jonathan Cornford

Arna Ghosh

Gauthier Gidel

Guillaume Lajoie

Blake Richards

Lire l'article

α-ReQ: Assessing Representation Quality in SSL

29 août 2023

α-ReQ : Évaluation de la qualité des représentations en apprentissage auto-supervisé

par

KK Agrawal

Arnab Kumar-Mondal

Arna Ghosh

Blake A. Richards

Lire l'article

Publications

Catalyzing next-generation Artificial Intelligence through NeuroAI

Anthony Zador

Sean Escola

Bence Ölveczky

Yoshua Bengio

Kwabena Boahen

Matthew Botvinick

Dmitri Chklovskii

Anne Churchland

Claudia Clopath

James DiCarlo

Surya

Surya Ganguli

Jeff Hawkins

Konrad Paul Kording

Alexei Koulakov

Yann LeCun

Timothy P. Lillicrap

Adam

Adam Marblestone … (voir 9 de plus)

Bruno Olshausen

Alexandre Pouget

Cristina Savin

Terrence Sejnowski

Eero Simoncelli

Sara Solla

David Sussillo

Andreas S. Tolias

Doris Tsao

2023-03-22

Nature Communications (publié)

Transfer Entropy Bottleneck: Learning Sequence to Sequence Information Transfer

Damjan Kalajdzievski

Ximeng Mao

Pascal Fortier-Poisson

When presented with a data stream of two statistically dependent variables, predicting the future of one of the variables (the target stream… (voir plus)) can benefit from information about both its history and the history of the other variable (the source stream). For example, fluctuations in temperature at a weather station can be predicted using both temperatures and barometric readings. However, a challenge when modelling such data is that it is easy for a neural network to rely on the greatest joint correlations within the target stream, which may ignore a crucial but small information transfer from the source to the target stream. As well, there are often situations where the target stream may have previously been modelled independently and it would be useful to use that model to inform a new joint model. Here, we develop an information bottleneck approach for conditional learning on two dependent streams of data. Our method, which we call Transfer Entropy Bottleneck (TEB), allows one to learn a model that bottlenecks the directed information transferred from the source variable to the target variable, while quantifying this information transfer within the model. As such, TEB provides a useful new information bottleneck approach for modelling two statistically dependent streams of data in order to make predictions about one of them.

2023-03-08

TMLR (accepté)

openreview.net

How gradient estimator variance and bias impact learning in neural networks

Arna Ghosh

Yuhan Helena Liu

Konrad Paul Kording

There is growing interest in understanding how real brains may approximate gradients and how gradients can be used to train neuromorphic chi… (voir plus)ps. However, neither real brains nor neuromorphic chips can perfectly follow the loss gradient, so parameter updates would necessarily use gradient estimators that have some variance and/or bias. Therefore, there is a need to understand better how variance and bias in gradient estimators impact learning dependent on network and task properties. Here, we show that variance and bias can impair learning on the training data, but some degree of variance and bias in a gradient estimator can be beneficial for generalization. We find that the ideal amount of variance and bias in a gradient estimator are dependent on several properties of the network and task: the size and activity sparsity of the network, the norm of the gradient, and the curvature of the loss landscape. As such, whether considering biologically-plausible learning algorithms or algorithms for training neuromorphic chips, researchers can analyze these properties to determine whether their approximation to gradient descent will be effective for learning given their network and task properties.

2023-02-01

ICLR.cc/2023/Conference (poster)

openreview.net

Formalizing locality for normative synaptic plasticity models

Colin Bredenberg

Ezekiel Williams

Cristina Savin

openreview.net

H OW GRADIENT ESTIMATOR VARIANCE AND BIAS COULD IMPACT LEARNING IN NEURAL CIRCUITS

Arna Ghosh

Yuhan Helena Liu

Konrad K¨ording

2023-01-01

(publié)

www.semanticscholar.org

Stimulus information guides the emergence of behavior related signals in primary somatosensory cortex during learning

Mariangela Panniello

Colleen J Gillon

Roberto Maffulli

Marco Celotto

Stefano Panzeri

Michael M Kohl

2022-12-05

bioRxiv (prépublication)

Adult neurogenesis acts as a neural regularizer

Lina M. Tran

Adam Santoro

Lulu Liu

Sheena A. Josselyn

Paul W. Frankland

2022-11-02

Proceedings of the National Academy of Sciences of the United States of America (publié)

A Generalized Bootstrap Target for Value-Learning, Efficiently Combining Value and Feature Predictions

Anthony GX-Chen

Veronica Chelu

Joelle Pineau

2022-06-28

Proceedings of the AAAI Conference on Artificial Intelligence (publié)

Towards Scaling Difference Target Propagation by Learning Backprop Targets

Maxence Ernoult

Fabrice Normandin

Abhinav Moudgil

Sean Spinney

The development of biologically-plausible learning algorithms is important for understanding learning in the brain, but most of them fail to… (voir plus) scale-up to real-world tasks, limiting their potential as explanations for learning by real brains. As such, it is important to explore learning algorithms that come with strong theoretical guarantees and can match the performance of backpropagation (BP) on complex tasks. One such algorithm is Difference Target Propagation (DTP), a biologically-plausible learning algorithm whose close relation with Gauss-Newton (GN) optimization has been recently established. However, the conditions under which this connection rigorously holds preclude layer-wise training of the feedback pathway synaptic weights (which is more biologically plausible). Moreover, good alignment between DTP weight updates and loss gradients is only loosely guaranteed and under very specific conditions for the architecture being trained. In this paper, we propose a novel feedback weight training scheme that ensures both that DTP approximates BP and that layer-wise feedback weight training can be restored without sacrificing any theoretical guarantees. Our theory is corroborated by experimental results and we report the best performance ever achieved by DTP on CIFAR-10 and ImageNet 32

2022-06-28

Proceedings of the 39th International Conference on Machine Learning (publié)

proceedings.mlr.press

On Neural Architecture Inductive Biases for Relational Tasks

Giancarlo Kerg

Sarthak Mittal

Current deep learning approaches have shown good in-distribution generalization performance, but struggle with out-of-distribution generaliz… (voir plus)ation. This is especially true in the case of tasks involving abstract relations like recognizing rules in sequences, as we find in many intelligence tests. Recent work has explored how forcing relational representations to remain distinct from sensory representations, as it seems to be the case in the brain, can help artificial systems. Building on this work, we further explore and formalize the advantages afforded by 'partitioned' representations of relations and sensory details, and how this inductive bias can help recompose learned relational structure in newly encountered settings. We introduce a simple architecture based on similarity scores which we name Compositional Relational Network (CoRelNet). Using this model, we investigate a series of inductive biases that ensure abstract relations are learned and represented distinctly from sensory data, and explore their effects on out-of-distribution generalization for a series of relational psychophysics tasks. We find that simple architectural choices can outperform existing models in out-of-distribution generalization. Together, these results show that partitioning relational representations from other information streams may be a simple way to augment existing network architectures' robustness when performing out-of-distribution relational computations.

2022-06-09

ArXiv (prépublication)

Evaluating Multimodal Interactive Agents

Josh Abramson

Arun Ahuja

Federico Carnevale

Petko Georgiev

Alex Goldin

Alden Hung

Jessica Landon

Timothy P. Lillicrap

Alistair M. Muldal

Adam Santoro

Tamara von Glehn

Greg Wayne

Nathaniel Wong

Chen Yan

Creating agents that can interact naturally with humans is a common goal in artificial intelligence (AI) research. However, evaluating these… (voir plus) interactions is challenging: collecting online human-agent interactions is slow and expensive, yet faster proxy metrics often do not correlate well with interactive evaluation. In this paper, we assess the merits of these existing evaluation metrics and present a novel approach to evaluation called the Standardised Test Suite (STS). The STS uses behavioural scenarios mined from real human interaction data. Agents see replayed scenario context, receive an instruction, and are then given control to complete the interaction offline. These agent continuations are recorded and sent to human annotators to mark as success or failure, and agents are ranked according to the proportion of continuations in which they succeed. The resulting STS is fast, controlled, interpretable, and representative of naturalistic interactions. Altogether, the STS consolidates much of what is desirable across many of our standard evaluation metrics, allowing us to accelerate research progress towards producing agents that can interact naturally with humans. A video may be found at https://youtu.be/YR1TngGORGQ.

2022-05-26

ArXiv (prépublication)

Current State and Future Directions for Learning in Biological Recurrent Neural Networks: A Perspective Piece

Luke Y. Prince

Roy Henha Eyono

Ellen Boven

Arna Ghosh

Joseph Pemberton

Franz Scherr

Claudia Clopath

Rui Ponte Costa

Wolfgang Maass

Cristina Savin

Katharina Wilmes

We provide a brief review of the common assumptions about biological learning with findings from experimental neuroscience and contrast them… (voir plus) with the efficiency of gradient-based learning in recurrent neural networks. The key issues discussed in this review include: synaptic plasticity, neural circuits, theory-experiment divide, and objective functions. We conclude with recommendations for both theoretical and experimental neuroscientists when designing new studies that could help bring clarity to these issues.

2022-04-27

Neurons, Behavior, Data analysis, and Theory (publié)