Blake Richards

ali.sahebpasand@mila.quebec

Ali Saheb Pasand

PhD - McGill University

Principal supervisor :

Pouya Bashivan

Alissia Di Maria

Research Intern - McGill University

alissia.dimaria@mila.quebec

Ann Huang

Collaborating Alumni

zixiang.huang@mila.quebec

Arna Ghosh

PhD - McGill University

Chen Chen

Postdoctorate - Université de Montréal

Principal supervisor :

Yoshua Bengio

chen.sun@mila.quebec

Colin Bredenberg

Postdoctorate - Université de Montréal

Principal supervisor :

Guillaume Lajoie

colin.bredenberg@mila.quebec

Dane Malenfant

Master's Research - McGill University

dane.malenfant@mila.quebec

daniel.levenstein@mila.quebec

Daniel Levenstein

Postdoctorate - McGill University

divyansha.lachi@mila.quebec

Divyansha Lachi

Collaborating researcher - Georgia Tech

Website

erica.cianfarano@mila.quebec

Dongyan Lin

PhD - McGill University

lindongy@mila.quebec

Erica Cianfarano

PhD - McGill University

Ethan Caballero

PhD - McGill University

Co-supervisor :

Irina Rish

ethan.caballero@mila.quebec

Website

etienne.barou-laforie@mila.quebec

Etienne Barou-Laforie

Research Intern - McGill University

Jonathan Cornford

Postdoctorate - McGill University

cornforj@mila.quebec

Josh Tindall

Master's Research - McGill University

joshua.tindall@mila.quebec

Mandana Samiei

PhD - McGill University

Principal supervisor :

PhD - McGill University

Mohammad Yaghoubi

PhD - McGill University

yaghoubm@mila.quebec

Roman.Pogodin@mila.Quebec

Quinn Lee

Postdoctorate - McGill University

j.quinn.lee@mila.quebec

Ray Chua

PhD - McGill University

Principal supervisor :

Doina Precup

chuaraym@mila.quebec

Roman Pogodin

Postdoctorate - McGill University

Co-supervisor :

Guillaume Lajoie

PhD - McGill University

roy.eyono@mila.quebec

santiago.jaramillo@mila.quebec

Santiago Jaramillo

Independent visiting researcher - University of Oregon

Website

vemund.schoyen@mila.quebec

Sonia Joseph

PhD - McGill University

sonia.joseph@mila.quebec

Collaborating Alumni

Research Intern - University of Oslo

Krystal Pan

Master's Research - McGill University

xuejing.pan@mila.quebec

Blog Posts

June 13, 2024

What Do Synaptic Weight Distributions Tell Us About Learning in the Brain ?

Roman Pogodin

Jonathan Cornford

Arna Ghosh

Gauthier Gidel

Guillaume Lajoie

Blake Richards

Read the article

August 29, 2023

α-ReQ: Assessing Representation Quality in SSL

KK Agrawal

Arnab Kumar-Mondal

Arna Ghosh

Blake A. Richards

Read the article

Publications

Adult neurogenesis acts as a neural regularizer

Lina M. Tran

Adam Santoro

Lulu Liu

Sheena A. Josselyn

Paul W. Frankland

2022-11-02

Proceedings of the National Academy of Sciences of the United States of America (published)

A Generalized Bootstrap Target for Value-Learning, Efficiently Combining Value and Feature Predictions

Anthony GX-Chen

Veronica Chelu

Joelle Pineau

2022-06-28

Proceedings of the AAAI Conference on Artificial Intelligence (published)

Towards Scaling Difference Target Propagation by Learning Backprop Targets

Maxence Ernoult

Fabrice Normandin

Abhinav Moudgil

Sean Spinney

The development of biologically-plausible learning algorithms is important for understanding learning in the brain, but most of them fail to… (see more) scale-up to real-world tasks, limiting their potential as explanations for learning by real brains. As such, it is important to explore learning algorithms that come with strong theoretical guarantees and can match the performance of backpropagation (BP) on complex tasks. One such algorithm is Difference Target Propagation (DTP), a biologically-plausible learning algorithm whose close relation with Gauss-Newton (GN) optimization has been recently established. However, the conditions under which this connection rigorously holds preclude layer-wise training of the feedback pathway synaptic weights (which is more biologically plausible). Moreover, good alignment between DTP weight updates and loss gradients is only loosely guaranteed and under very specific conditions for the architecture being trained. In this paper, we propose a novel feedback weight training scheme that ensures both that DTP approximates BP and that layer-wise feedback weight training can be restored without sacrificing any theoretical guarantees. Our theory is corroborated by experimental results and we report the best performance ever achieved by DTP on CIFAR-10 and ImageNet 32

2022-06-28

Proceedings of the 39th International Conference on Machine Learning (published)

proceedings.mlr.press

On Neural Architecture Inductive Biases for Relational Tasks

Giancarlo Kerg

Sarthak Mittal

Current deep learning approaches have shown good in-distribution generalization performance, but struggle with out-of-distribution generaliz… (see more)ation. This is especially true in the case of tasks involving abstract relations like recognizing rules in sequences, as we find in many intelligence tests. Recent work has explored how forcing relational representations to remain distinct from sensory representations, as it seems to be the case in the brain, can help artificial systems. Building on this work, we further explore and formalize the advantages afforded by 'partitioned' representations of relations and sensory details, and how this inductive bias can help recompose learned relational structure in newly encountered settings. We introduce a simple architecture based on similarity scores which we name Compositional Relational Network (CoRelNet). Using this model, we investigate a series of inductive biases that ensure abstract relations are learned and represented distinctly from sensory data, and explore their effects on out-of-distribution generalization for a series of relational psychophysics tasks. We find that simple architectural choices can outperform existing models in out-of-distribution generalization. Together, these results show that partitioning relational representations from other information streams may be a simple way to augment existing network architectures' robustness when performing out-of-distribution relational computations.

2022-06-09

ArXiv (preprint)

Evaluating Multimodal Interactive Agents

Josh Abramson

Arun Ahuja

Federico Carnevale

Petko Georgiev

Alex Goldin

Alden Hung

Jessica Landon

Timothy P. Lillicrap

Alistair M. Muldal

Adam Santoro

Tamara von Glehn

Greg Wayne

Nathaniel Wong

Chen Yan

Creating agents that can interact naturally with humans is a common goal in artificial intelligence (AI) research. However, evaluating these… (see more) interactions is challenging: collecting online human-agent interactions is slow and expensive, yet faster proxy metrics often do not correlate well with interactive evaluation. In this paper, we assess the merits of these existing evaluation metrics and present a novel approach to evaluation called the Standardised Test Suite (STS). The STS uses behavioural scenarios mined from real human interaction data. Agents see replayed scenario context, receive an instruction, and are then given control to complete the interaction offline. These agent continuations are recorded and sent to human annotators to mark as success or failure, and agents are ranked according to the proportion of continuations in which they succeed. The resulting STS is fast, controlled, interpretable, and representative of naturalistic interactions. Altogether, the STS consolidates much of what is desirable across many of our standard evaluation metrics, allowing us to accelerate research progress towards producing agents that can interact naturally with humans. A video may be found at https://youtu.be/YR1TngGORGQ.

2022-05-26

ArXiv (preprint)

Current State and Future Directions for Learning in Biological Recurrent Neural Networks: A Perspective Piece

Luke Y. Prince

Roy Henha Eyono

Ellen Boven

Arna Ghosh

Joseph Pemberton

Franz Scherr

Claudia Clopath

Rui Ponte Costa

Wolfgang Maass

Cristina Savin

Katharina Wilmes

We provide a brief review of the common assumptions about biological learning with findings from experimental neuroscience and contrast them… (see more) with the efficiency of gradient-based learning in recurrent neural networks. The key issues discussed in this review include: synaptic plasticity, neural circuits, theory-experiment divide, and objective functions. We conclude with recommendations for both theoretical and experimental neuroscientists when designing new studies that could help bring clarity to these issues.

2022-04-27

Neurons, Behavior, Data analysis, and Theory (published)

Can neurogenesis act as a neural regularizer?

Lina M. Tran

Adam Santoro

Lulu Liu

Sheena A. Josselyn

Paul W. Frankland

New neurons are continuously generated in the subgranular zone of the dentate gyrus throughout adulthood. These new neurons gradually integr… (see more)ate into hippocampal circuits, forming new naïve synapses. Viewed from this perspective, these new neurons may represent a significant source of ‘wiring’ noise in hippocampal networks. In machine learning, such noise injection is commonly used as a regularization technique. Regularization techniques help prevent overfitting training data, and allow models to generalize learning to new, unseen data. Using a computational modeling approach, here we ask whether a neurogenesis-like process similarly acts as a regularizer, facilitating generalization in a category learning task. In a convolutional neural network (CNN) trained on the CIFAR-10 object recognition dataset, we modeled neurogenesis as a replacement/turnover mechanism, where weights for a randomly chosen small subset of neurons in a chosen hidden layer were re-initialized to new values as the model learned to categorize 10 different classes of objects. We found that neurogenesis enhanced generalization on unseen test data compared to networks with no neurogenesis. Moreover, neurogenic networks either outperformed or performed similarly to networks with conventional noise injection (i.e., dropout, weight decay, and neural noise). These results suggest that neurogenesis can enhance generalization in hippocampal learning through noise-injection, expanding on the roles that neurogenesis may have in cognition. Author Summary In deep neural networks, various forms of noise injection are used as regularization techniques to prevent overfitting and promote generalization on unseen test data. Here, we were interested in whether adult neurogenesis– the lifelong production of new neurons in the hippocampus– might similarly function as a regularizer in the brain. We explored this question computationally, assessing whether implementing a neurogenesis-like process in a hidden layer within a convolutional neural network trained in a category learning task would prevent overfitting and promote generalization. We found that neurogenesis regularization was as least as effective as, or more effective than, conventional regularizers (i.e., dropout, weight decay and neural noise) in improving model performance. These results suggest that optimal levels of hippocampal neurogenesis may improve memory-guided decision making by preventing overfitting, thereby promoting the formation of more generalized memories that can be applied in a broader range of circumstances. We outline how these predictions may be evaluated behaviorally in rodents with altered hippocampal neurogenesis.

2022-04-10

bioRxiv (preprint)

Forgetting Enhances Episodic Control With Structured Memories

Annik Yalnizyan-Carson

Forgetting is a normal process in healthy brains, and evidence suggests that the mammalian brain forgets more than is required based on limi… (see more)tations of mnemonic capacity. Episodic memories, in particular, are liable to be forgotten over time. Researchers have hypothesized that it may be beneficial for decision making to forget episodic memories over time. Reinforcement learning offers a normative framework in which to test such hypotheses. Here, we show that a reinforcement learning agent that uses an episodic memory cache to find rewards in maze environments can forget a large percentage of older memories without any performance impairments, if they utilize mnemonic representations that contain structural information about space. Moreover, we show that some forgetting can actually provide a benefit in performance compared to agents with unbounded memories. Our analyses of the agents show that forgetting reduces the influence of outdated information and states which are not frequently visited on the policies produced by the episodic control system. These results support the hypothesis that some degree of forgetting can be beneficial for decision making, which can help to explain why the brain forgets more than is required by capacity limitations.

2022-03-25

Frontiers in Computational Neuroscience (published)

Inductive Biases for Relational Tasks

Giancarlo Kerg

Sarthak Mittal

Current deep learning approaches have shown good in-distribution performance but struggle in out-of-distribution settings. This is especiall… (see more)y true in the case of tasks involving abstract relations like recognizing rules in sequences, as required in many intelligence tests. In contrast, our brains are remarkably flexible at such tasks, an attribute that is likely linked to anatomical constraints on computations. Inspired by this, recent work has explored how enforcing that relational representations remain distinct from sensory representations can help artificial systems. Building on this work, we further explore and formalize the advantages afforded by ``partitioned'' representations of relations and sensory details. We investigate inductive biases that ensure abstract relations are learned and represented distinctly from sensory data across several neural network architectures and show that they outperform existing architectures on out-of-distribution generalization for various relational tasks. These results show that partitioning relational representations from other information streams may be a simple way to augment existing network architectures' robustness when performing relational computations.

2022-03-25

ICLR.cc/2022/Workshop/OSC (poster)

openreview.net

The Brain-Computer Metaphor Debate Is Useless: A Matter of Semantics

Timothy P. Lillicrap

2022-02-08

Frontiers in Computer Science (published)

A Generalized Bootstrap Target for Value-Learning, Efficiently Combining Value and Feature Predictions

Anthony GX-Chen

Veronica Chelu

Joelle Pineau

Estimating value functions is a core component of reinforcement learning algorithms. Temporal difference (TD) learning algorithms use bootst… (see more)rapping, i.e. they update the value function toward a learning target using value estimates at subsequent time-steps. Alternatively, the value function can be updated toward a learning target constructed by separately predicting successor features (SF)—a policy-dependent model—and linearly combining them with instantaneous rewards. We focus on bootstrapping targets used when estimating value functions, and propose a new backup target, the ?-return mixture, which implicitly combines value-predictive knowledge (used by TD methods) with (successor) feature-predictive knowledge—with a parameter ? capturing how much to rely on each. We illustrate that incorporating predictive knowledge through an ??-discounted SF model makes more efficient use of sampled experience, compared to either extreme, i.e. bootstrapping entirely on the value function estimate, or bootstrapping on the product of separately estimated successor features and instantaneous reward models. We empirically show this approach leads to faster policy evaluation and better control performance, for tabular and nonlinear function approximations, indicating scalability and generality.

2022-01-05

ArXiv (preprint)

$\alpha$-ReQ : Assessing Representation Quality in Self-Supervised Learning by measuring eigenspectrum decay

Kumar Krishna Agrawal

Arnab Kumar Mondal

Arna Ghosh