Eugene Belilovsky

nicolas.bernier@mila.quebec

nader.asadi@mila.quebec

Master's Research - Concordia University

mohammadreza.davari@mila.quebec

Reza Davari

PhD - Concordia University

Research Intern - Concordia University

louis.fournier@mila.quebec

Master's Research - Concordia University

alexander.fulleringer@mila.quebec

Paul Janson

Master's Research - Concordia University

paul.janson@mila.quebec

Charles-Etienne Joseph

Master's Research - Université de Montréal

Co-supervisor :

charles-etienne.joseph@mila.quebec

Zafir Khalid

Master's Research - Concordia University

zafir.khaled@mila.quebec

Gwen Legate

PhD - Concordia University

Co-supervisor :

gwendolyne.legate@mila.quebec

abhinav.moudgil@mila.quebec

Abhinav Moudgil

PhD - Concordia University

Adel Nabli

PhD - Concordia University

adel.nabli@mila.quebec

Geraldin Nanfack

Postdoctorate - Concordia University

Co-supervisor :

geraldin.nanfack@mila.quebec

Albert Orozco Camacho

PhD - Concordia University

Co-supervisor :

Master's Research - Concordia University

Co-supervisor :

paria.mehrbod@mila.quebec

Donald Shenaj

Collaborating researcher - Concordia University

Co-supervisor :

donald.shenaj@mila.quebec

PhD - Concordia University

Co-supervisor :

vaibhav.singh@mila.quebec

Benjamin Therien

PhD - Université de Montréal

Principal supervisor :

benjamin.therien@mila.quebec

Collaborating researcher - Université de Montréal

Principal supervisor :

pedro.vianna@mila.quebec

Humza Wajid Hameed

Master's Research - Concordia University

humza.wajid@mila.quebec

amirhossein.zamani@mila.quebec

AmirHossein Zamani

PhD - Concordia University

Congshu Zou

Master's Research - Concordia University

congshu.zou@mila.quebec

Publications

New Insights on Reducing Abrupt Representation Change in Online Continual Learning

Lucas Caccia

Rahaf Aljundi

Nader Asadi

Tinne Tuytelaars

Joelle Pineau

In the online continual learning paradigm, agents must learn from a changing distribution while respecting memory and compute constraints. E… (see more)xperience Replay (ER), where a small subset of past data is stored and replayed alongside new data, has emerged as a simple and effective learning strategy. In this work, we focus on the change in representations of observed data that arises when previously unobserved classes appear in the incoming data stream, and new classes must be distinguished from previous ones. We shed new light on this question by showing that applying ER causes the newly added classes’ representations to overlap significantly with the previous classes, leading to highly disruptive parameter updates. Based on this empirical analysis, we propose a new method which mitigates this issue by shielding the learned representations from drastic adaptation to accommodate new classes. We show that using an asymmetric update rule pushes new classes to adapt to the older ones (rather than the reverse), which is more effective especially at task boundaries, where much of the forgetting typically occurs. Empirical results show significant gains over strong baselines on standard continual learning benchmarks.

2022-01-28

ICLR.cc/2022/Conference (poster)

doi.org

openreview.net

Parametric Scattering Networks

Shanel Gauthier

Benjamin Th'erien

Laurent Alséne-Racicot

Michael Eickenberg

The wavelet scattering transform creates geometric in-variants and deformation stability. In multiple signal do-mains, it has been shown to … (see more)yield more discriminative rep-resentations compared to other non-learned representations and to outperform learned representations in certain tasks, particularly on limited labeled data and highly structured signals. The wavelet filters used in the scattering trans-form are typically selected to create a tight frame via a pa-rameterized mother wavelet. In this work, we investigate whether this standard wavelet filterbank construction is op-timal. Focusing on Morlet wavelets, we propose to learn the scales, orientations, and aspect ratios of the filters to produce problem-specific parameterizations of the scattering transform. We show that our learned versions of the scattering transform yield significant performance gains in small-sample classification settings over the standard scat-tering transform. Moreover, our empirical results suggest that traditional filterbank constructions may not always be necessary for scattering transforms to extract effective rep-resentations.

2021-07-20

ArXiv (preprint)

doi.org

arxiv.org

Online Continual Learning with Maximally Interfered Retrieval

Rahaf Aljundi

Lucas Caccia

Massimo Caccia

Min Lin

Laurent Charlin

Tinne Tuytelaars

Continual learning, the setting where a learning agent is faced with a never ending stream of data, continues to be a great challenge for mo… (see more)dern machine learning systems. In particular the online or "single-pass through the data" setting has gained attention recently as a natural setting that is difficult to tackle. Methods based on replay, either generative or from a stored memory, have been shown to be effective approaches for continual learning, matching or exceeding the state of the art in a number of standard benchmarks. These approaches typically rely on randomly selecting samples from the replay memory or from a generative model, which is suboptimal. In this work, we consider a controlled sampling of memories for replay. We retrieve the samples which are most interfered, i.e. whose prediction will be most negatively impacted by the foreseen parameters update. We show a formulation for this sampling criterion in both the generative replay and the experience replay setting, producing consistent gains in performance and greatly reduced forgetting. We release an implementation of our method at this https URL.

2019-08-11

ArXiv (preprint)

arxiv.org

Learning Optimizers for Local SGD

Charles-Étienne Joseph

Benjamin Thérien

Abhinav Moudgil

Boris Knyazev