Publications

An Empirical Study of Example Forgetting During Deep Neural Network Learning
Mariya Toneva
Alessandro Sordoni
Remi Tachet des Combes
Adam Trischler
Yoshua Bengio
Geoffrey J. Gordon
Inspired by the phenomenon of catastrophic forgetting, we investigate the learning dynamics of neural networks as they train on single class… (voir plus)ification tasks. Our goal is to understand whether a related phenomenon occurs when data does not undergo a clear distributional shift. We define a `forgetting event' to have occurred when an individual training example transitions from being classified correctly to incorrectly over the course of learning. Across several benchmark data sets, we find that: (i) certain examples are forgotten with high frequency, and some not at all; (ii) a data set's (un)forgettable examples generalize across neural architectures; and (iii) based on forgetting dynamics, a significant fraction of examples can be omitted from the training data set while still maintaining state-of-the-art generalization performance.
2018-12-31
ICLR.cc/2019/Conference (poster)
doi.org
openreview.net
A Geometric Perspective on Optimal Representations for Reinforcement Learning
Bellemare Marc-Emmanuel
Will Dabney
Robert Dadashi
Adrien Ali Taiga
Pablo Samuel Castro
Nicolas Roux
Dale Schuurmans
Tor Lattimore
Clare Lyle
2018-12-31
NeurIPS (publié)
openreview.net
openreview.net
Gradient based sample selection for online continual learning
Rahaf Aljundi
Min Lin
Baptiste Goujaud
Yoshua Bengio
A continual learning agent learns online with a non-stationary and never-ending stream of data. The key to such learning process is to overc… (voir plus)ome the catastrophic forgetting of previously seen data, which is a well known problem of neural networks. To prevent forgetting, a replay buffer is usually employed to store the previous data for the purpose of rehearsal. Previous works often depend on task boundary and i.i.d. assumptions to properly select samples for the replay buffer. In this work, we formulate sample selection as a constraint reduction problem based on the constrained optimization view of continual learning. The goal is to select a fixed subset of constraints that best approximate the feasible region defined by the original constraints. We show that it is equivalent to maximizing the diversity of samples in the replay buffer with parameters gradient as the feature. We further develop a greedy alternative that is cheap and efficient. The advantage of the proposed method is demonstrated by comparing to other alternatives under the continual learning setting. Further comparisons are made against state of the art methods that rely on task boundaries which show comparable or even better results for our method.
2018-12-31
NeurIPS (publié)
arxiv.org
arxiv.org
h-detach: Modifying the LSTM Gradient Towards Better Optimization
Devansh Arpit
Bhargav Kanuparthi
Giancarlo Kerg
Nan Rosemary Ke
Ioannis Mitliagkas
Yoshua Bengio
Recurrent neural networks are known for their notorious exploding and vanishing gradient problem (EVGP). This problem becomes more evident i… (voir plus)n tasks where the information needed to correctly solve them exist over long time scales, because EVGP prevents important gradient components from being back-propagated adequately over a large number of steps. We introduce a simple stochastic algorithm (\textit{h}-detach) that is specific to LSTM optimization and targeted towards addressing this problem. Specifically, we show that when the LSTM weights are large, the gradient components through the linear path (cell state) in the LSTM computational graph get suppressed. Based on the hypothesis that these components carry information about long term dependencies (which we show empirically), their suppression can prevent LSTMs from capturing them. Our algorithm\footnote{Our code is available at this https URL.} prevents gradients flowing through this path from getting suppressed, thus allowing the LSTM to capture such dependencies better. We show significant improvements over vanilla LSTM gradient based training in terms of convergence speed, robustness to seed and learning rate, and generalization using our modification of LSTM gradient on various benchmark datasets.
2018-12-31
ICLR.cc/2019/Conference (poster)
openreview.net
openreview.net
Hierarchical Importance Weighted Autoencoders
Chin-wei Huang
Kris Sankaran
Eeshan Dhekane
Alexandre Lacoste
Aaron Courville
Importance weighted variational inference (Burda et al., 2015) uses multiple i.i.d. samples to have a tighter variational lower bound. We be… (voir plus)lieve a joint proposal has the potential of reducing the number of redundant samples, and introduce a hierarchical structure to induce correlation. The hope is that the proposals would coordinate to make up for the error made by one another to reduce the variance of the importance estimator. Theoretically, we analyze the condition under which convergence of the estimator variance can be connected to convergence of the lower bound. Empirically, we confirm that maximization of the lower bound does implicitly minimize variance. Further analysis shows that this is a result of negative correlation induced by the proposed hierarchical meta sampling scheme, and performance of inference also improves when the number of samples increases.
2018-12-31
ICML (publié)
proceedings.mlr.press
proceedings.mlr.press
Hindsight Credit Assignment
Anna Harutyunyan
Will Dabney
Thomas Mesnard
Mohammad Gheshlaghi Azar
Bilal Piot
Nicolas Heess
Hado van Hasselt
Greg Wayne
Satinder Singh
Doina Precup
Remi Munos
2018-12-31
Advances in Neural Information Processing Systems 32 (NeurIPS 2019) (publié)
arxiv.org
arxiv.org
How to Initialize your Network? Robust Initialization for WeightNorm & ResNets
Devansh Arpit
Vı́ctor Campos
Yoshua Bengio
Residual networks (ResNet) and weight normalization play an important role in various deep learning applications. However, parameter initial… (voir plus)ization strategies have not been studied previously for weight normalized networks and, in practice, initialization methods designed for un-normalized networks are used as a proxy. Similarly, initialization for ResNets have also been studied for un-normalized networks and often under simplified settings ignoring the shortcut connection. To address these issues, we propose a novel parameter initialization strategy that avoids explosion/vanishment of information across layers for weight normalized networks with and without residual connections. The proposed strategy is based on a theoretical analysis using mean field approximation. We run over 2,500 experiments and evaluate our proposal on image datasets showing that the proposed initialization outperforms existing initialization methods in terms of generalization performance, robustness to hyper-parameter values and variance between seeds, especially when networks get deeper in which case existing methods fail to even start training. Finally, we show that using our initialization in conjunction with learning rate warmup is able to reduce the gap between the performance of weight normalized and batch normalized networks.
2018-12-31
NeurIPS (publié)
arxiv.org
arxiv.org
Improving advance medical directives: lessons from Quebec
Louise G. Bernier
Catherine Régis
Policy-makers’ efforts to increase the uptake of advance medical directives (AMDs), and the legal constraints they impose on health profes… (voir plus)sionals, are bringing greater scrutiny to provincial AMD regimes. In 2015, Quebec introduced a new, legally binding form to be filled out for AMDs, which limits individuals’ expression of their wishes to narrow, checklist responses to questions on specific medical interventions. This form-focused regime has other shortcomings: it relies on individuals to self-inform and it does not provide them the opportunity to meaningfully convey their preferences for end-of-life care. A more values-based and collaborative approach provides a better path forward for Quebec and for other provinces.
2018-12-31
(publié)
www.semanticscholar.org
InfoBot: Structured Exploration in ReinforcementLearning Using Information Bottleneck
Anirudh Goyal
Riashat Islam
D. Strouse
Zafarali Ahmed
Matthew Botvinick
Hugo Larochelle
Yoshua Bengio
Sergey Levine
2018-12-31
(publié)
www.semanticscholar.org
InfoBot: Transfer and Exploration via the Information Bottleneck
Anirudh Goyal
Riashat Islam
Daniel Strouse
Zafarali Ahmed
Matthew Botvinick
Hugo Larochelle
Sergey Levine
Yoshua Bengio
A central challenge in reinforcement learning is discovering effective policies for tasks where rewards are sparsely distributed. We postula… (voir plus)te that in the absence of useful reward signals, an effective exploration strategy should seek out {\it decision states}. These states lie at critical junctions in the state space from where the agent can transition to new, potentially unexplored regions. We propose to learn about decision states from prior experience. By training a goal-conditioned policy with an information bottleneck, we can identify decision states by examining where the model actually leverages the goal state. We find that this simple mechanism effectively identifies decision states, even in partially observed settings. In effect, the model learns the sensory cues that correlate with potential subgoals. In new environments, this model can then identify novel subgoals for further exploration, guiding the agent through a sequence of potential decision states and through new regions of the state space.
2018-12-31
ICLR.cc/2019/Conference (poster)
doi.org
openreview.net
Interpolated Adversarial Training: Achieving Robust Neural Networks without Sacrificing Accuracy
Alex Lamb
Vikas Verma
Juho Kannala
Yoshua Bengio
Adversarial robustness has become a central goal in deep learning, both in theory and practice. However, successful methods to improve adver… (voir plus)sarial robustness (such as adversarial training) greatly hurt generalization performance on the clean data. This could have a major impact on how adversarial robustness affects real world systems (i.e. many may opt to forego robustness if it can improve performance on the clean data). We propose Interpolated Adversarial Training, which employs recently proposed interpolation based training methods in the framework of adversarial training. On CIFAR-10, adversarial training increases clean test error from 5.8% to 16.7%, whereas with our Interpolated adversarial training we retain adversarial robustness while achieving a clean test error of only 6.5%. With our technique, the relative error increase for the robust model is reduced from 187.9% to just 12.1%.
2018-12-31
arXiv.org (prépublication)
dblp.uni-trier.de
Lagrangian neurodynamics for real-time error-backpropagation across cortical areas
Dominik Dold
Akos F. Kungl
João Sacramento
Mihai A. Petrovici
Kaspar Anton Schindler
Jonathan Binas
Yoshua Bengio
Walter Senn
.
2018-12-31
(publié)
www.semanticscholar.org