Portrait of Fabrice Normandin

Fabrice Normandin

normandf@mila.quebec
Developer, Research Software, Innovation, Development and Technologies

Publications

The oneirogen hypothesis: modeling the hallucinatory effects of classical psychedelics in terms of replay-dependent plasticity mechanisms
Colin Bredenberg
Fabrice Normandin
Blake Richards
Guillaume Lajoie
2024-09-30
bioRxiv (preprint)
doi.org
Towards Scaling Difference Target Propagation by Learning Backprop Targets
Maxence Ernoult
Fabrice Normandin
Abhinav Moudgil
Sean Spinney
Eugene Belilovsky
Irina Rish
Blake Richards
Yoshua Bengio
The development of biologically-plausible learning algorithms is important for understanding learning in the brain, but most of them fail to… (see more) scale-up to real-world tasks, limiting their potential as explanations for learning by real brains. As such, it is important to explore learning algorithms that come with strong theoretical guarantees and can match the performance of backpropagation (BP) on complex tasks. One such algorithm is Difference Target Propagation (DTP), a biologically-plausible learning algorithm whose close relation with Gauss-Newton (GN) optimization has been recently established. However, the conditions under which this connection rigorously holds preclude layer-wise training of the feedback pathway synaptic weights (which is more biologically plausible). Moreover, good alignment between DTP weight updates and loss gradients is only loosely guaranteed and under very specific conditions for the architecture being trained. In this paper, we propose a novel feedback weight training scheme that ensures both that DTP approximates BP and that layer-wise feedback weight training can be restored without sacrificing any theoretical guarantees. Our theory is corroborated by experimental results and we report the best performance ever achieved by DTP on CIFAR-10 and ImageNet 32
2022-06-28
Proceedings of the 39th International Conference on Machine Learning (published)
proceedings.mlr.press
arxiv.org
Sequoia: A Software Framework to Unify Continual Learning Research
Fabrice Normandin
Oleksiy Ostapenko
Pau Rodriguez
Florian Golemo
Ryan Lindeborg
J. Hurtado
Matthew D Riemer
Lucas Cecchi
Timothee LESORT
Dominic Zhao
Khimya Khetarpal
David Vazquez
Laurent Charlin
Irina Rish
Massimo Caccia
The field of Continual Learning (CL) seeks to develop algorithms that accumulate knowledge and skills over time through interaction with non… (see more)-stationary environments. In practice, a plethora of evaluation procedures (settings) and algorithmic solutions (methods) exist, each with their own potentially disjoint set of assumptions. This variety makes measuring progress in CL difficult. We propose a taxonomy of settings, where each setting is described as a set of assumptions. A tree-shaped hierarchy emerges from this view, where more general settings become the parents of those with more restrictive assumptions. This makes it possible to use inheritance to share and reuse research, as developing a method for a given setting also makes it directly applicable onto any of its children. We instantiate this idea as a publicly available software framework called Sequoia, which features a wide variety of settings from both the Continual Supervised Learning (CSL) and Continual Reinforcement Learning (CRL) domains. Sequoia also includes a growing suite of methods which are easy to extend and customize, in addition to more specialized methods from external libraries. We hope that this new paradigm and its first implementation can help unify and accelerate research in CL. You can help us grow the tree by visiting (this GitHub URL).
2021-08-02
ArXiv (preprint)
openreview.net
openreview.net
Online Fast Adaptation and Knowledge Accumulation: a New Approach to Continual Learning
Massimo Caccia
Pau Rodriguez
Oleksiy Ostapenko
Fabrice Normandin
Min Lin
Lucas Caccia
Issam Hadj Laradji
Irina Rish
Alexande Lacoste
David Vazquez
Laurent Charlin
2020-03-12
ArXiv (preprint)
arxiv.org
arxiv.org
Online Fast Adaptation and Knowledge Accumulation (OSAKA): a New Approach to Continual Learning.
Massimo Caccia
Pau Rodriguez
Oleksiy Ostapenko
Fabrice Normandin
Min Lin
Lucas Caccia
Issam Hadj Laradji
Irina Rish
Alexandre Lacoste
David Vazquez
Laurent Charlin