Publications

Conditional Computation for Continual Learning
Min Lin
Jie Fu
Yoshua Bengio
Catastrophic forgetting of connectionist neural networks is caused by the global sharing of parameters among all training examples. In this … (see more)study, we analyze parameter sharing under the conditional computation framework where the parameters of a neural network are conditioned on each input example. At one extreme, if each input example uses a disjoint set of parameters, there is no sharing of parameters thus no catastrophic forgetting. At the other extreme, if the parameters are the same for every example, it reduces to the conventional neural network. We then introduce a clipped version of maxout networks which lies in the middle, i.e. parameters are shared partially among examples. Based on the parameter sharing analysis, we can locate a limited set of examples that are interfered when learning a new example. We propose to perform rehearsal on this set to prevent forgetting, which is termed as conditional rehearsal. Finally, we demonstrate the effectiveness of the proposed method in an online non-stationary setup, where updates are made after each new example and the distribution of the received example shifts over time.
2019-06-15
ArXiv (preprint)
arxiv.org
arxiv.org
Learning the Arrow of Time
Nasim Rahaman
Steffen Wolf
Anirudh Goyal
Roman Remme
Yoshua Bengio
We humans seem to have an innate understanding of the asymmetric progression of time, which we use to efficiently and safely perceive and ma… (see more)nipulate our environment. Drawing inspiration from that, we address the problem of learning an arrow of time in a Markov (Decision) Process. We illustrate how a learned arrow of time can capture meaningful information about the environment, which in turn can be used to measure reachability, detect side-effects and to obtain an intrinsic reward signal. We show empirical results on a selection of discrete and continuous environments, and demonstrate for a class of stochastic processes that the learned arrow of time agrees reasonably well with a known notion of an arrow of time given by the celebrated Jordan-Kinderlehrer-Otto result.
2019-06-11
OpenReview.net/Anonymous_Preprint (unknown)
openreview.net
openreview.net
Learning Powerful Policies by Using Consistent Dynamics Model
Shagun Sodhani
Anirudh Goyal
Tristan Deleu
Yoshua Bengio
Sergey Levine
Jian Tang
Model-based Reinforcement Learning approaches have the promise of being sample efficient. Much of the progress in learning dynamics models i… (see more)n RL has been made by learning models via supervised learning. But traditional model-based approaches lead to `compounding errors' when the model is unrolled step by step. Essentially, the state transitions that the learner predicts (by unrolling the model for multiple steps) and the state transitions that the learner experiences (by acting in the environment) may not be consistent. There is enough evidence that humans build a model of the environment, not only by observing the environment but also by interacting with the environment. Interaction with the environment allows humans to carry out experiments: taking actions that help uncover true causal relationships which can be used for building better dynamics models. Analogously, we would expect such interactions to be helpful for a learning agent while learning to model the environment dynamics. In this paper, we build upon this intuition by using an auxiliary cost function to ensure consistency between what the agent observes (by acting in the real world) and what it imagines (by acting in the `learned' world). We consider several tasks - Mujoco based control tasks and Atari games - and show that the proposed approach helps to train powerful policies and better dynamics models.
2019-06-10
ArXiv (preprint)
arxiv.org
arxiv.org
Note on the bias and variance of variational inference
Chin-wei Huang
Aaron Courville
In this note, we study the relationship between the variational gap and the variance of the (log) likelihood ratio. We show that the gap can… (see more) be upper bounded by some form of dispersion measure of the likelihood ratio, which suggests the bias of variational inference can be reduced by making the distribution of the likelihood ratio more concentrated, such as via averaging and variance reduction.
2019-06-08
ArXiv (preprint)
arxiv.org
arxiv.org
Dissociating memory accessibility and precision in forgetting
Sam C. Berens
Blake A. Richards
Aidan J. Horner
2019-06-03
Nature Human Behaviour (published)
doi.org
Learning Domain Randomization Distributions for Transfer of Locomotion Policies
Melissa Mozian
Juan Higuera
David Meger
Gregory Dudek
Domain randomization (DR) is a successful technique for learning robust policies for robot systems, when the dynamics of the target robot sy… (see more)stem are unknown. The success of policies trained with domain randomization however, is highly dependent on the correct selection of the randomization distribution. The majority of success stories typically use real world data in order to carefully select the DR distribution, or incorporate real world trajectories to better estimate appropriate randomization distributions. In this paper, we consider the problem of finding good domain randomization parameters for simulation, without prior access to data from the target system. We explore the use of gradient-based search methods to learn a domain randomization with the following properties: 1) The trained policy should be successful in environments sampled from the domain randomization distribution 2) The domain randomization distribution should be wide enough so that the experience similar to the target robot system is observed during training, while addressing the practicality of training finite capacity models. These two properties aim to ensure the trajectories encountered in the target system are close to those observed during training, as existing methods in machine learning are better suited for interpolation than extrapolation. We show how adapting the domain randomization distribution while training context-conditioned policies results in improvements on jump-start and asymptotic performance when transferring a learned policy to the target environment.
2019-06-01
ArXiv (preprint)
arxiv.org
arxiv.org
Dynamic spectrum access under partial observations: A restless bandit approach
Nima Akbarzadeh
Aditya Mahajan
We consider a communication system where multiple unknown channels are available for transmission. Each channel is a channel with state whic… (see more)h evolves in a Markov manner. The transmitter has to select L channels to use and also decide the resources (e.g., power, rate, etc.) to use for each of the selected channels. It observes the state of the channels it uses and receives no feedback on the state of the other channels. We model this problem as a partially observable Markov decision process and obtain a simplified belief state. We show that the optimal resource allocation policy can be identified in closed form. Once the optimal resource allocation policy is fixed, choosing the channel scheduling policy may be viewed as a restless bandit. We present an efficient algorithm to check indexability and compute the Whittle index for each channel. When the model is indexable, the Whittle index policy, which transmits over the L channels with the smallest Whittle indices, is an attractive heuristic policy.
2019-05-31
Canadian Workshop on Information Theory (published)
doi.org
Understanding the Behaviour of Neural Abstractive Summarizers using Contrastive Examples
Krtin Kumar
Jackie CK Cheung
Neural abstractive summarizers generate summary texts using a language model conditioned on the input source text, and have recently achieve… (see more)d high ROUGE scores on benchmark summarization datasets. We investigate how they achieve this performance with respect to human-written gold-standard abstracts, and whether the systems are able to understand deeper syntactic and semantic structures. We generate a set of contrastive summaries which are perturbed, deficient versions of human-written summaries, and test whether existing neural summarizers score them more highly than the human-written summaries. We analyze their performance on different datasets and find that these systems fail to understand the source text, in a majority of the cases.
2019-05-31
North American Chapter of the Association for Computational Linguistics (published)
doi.org
Human Motion Prediction Via Pattern Completion in Latent Representation Space
Yi Tian Xu
Yaqiao Li
David Meger
Inspired by ideas in cognitive science, we propose a novel and general approach to solve human motion understanding via pattern completion o… (see more)n a learned latent representation space. Our model outperforms current state-of-the-art methods in human motion prediction across a number of tasks, with no customization. To construct a latent representation for time-series of various lengths, we propose a new and generic autoencoder based on sequence-to-sequence learning. While traditional inference strategies find a correlation between an input and an output, we use pattern completion, which views the input as a partial pattern and to predict the best corresponding complete pattern. Our results demonstrate that this approach has advantages when combined with our autoencoder in solving human motion prediction, motion generation and action classification.
2019-05-28
2019 16th Conference on Computer and Robot Vision (CRV) (published)
doi.org
arxiv.org
Unsupervised Controllable Text Generation with Global Variation Discovery and Disentanglement
Peng Xu
Yanshuai Cao
Jackie CK Cheung
Existing controllable text generation systems rely on annotated attributes, which greatly limits their capabilities and applications. In thi… (see more)s work, we make the first successful attempt to use VAEs to achieve controllable text generation without supervision. We do so by decomposing the latent space of the VAE into two parts: one incorporates structural constraints to capture dominant global variations implicitly present in the data, e.g., sentiment or topic; the other is unstructured and is used for the reconstruction of the source sentences. With the enforced structural constraint, the underlying global variations will be discovered and disentangled during the training of the VAE. The structural constraint also provides a natural recipe for mitigating posterior collapse for the structured part, which cannot be fully resolved by the existing techniques. On the task of text style transfer, our unsupervised approach achieves significantly better performance than previous supervised approaches. By showcasing generation with finer-grained control including Cards-Against-Humanity-style topic transitions within a sentence, we demonstrate that our model can perform controlled text generation in a more flexible way than existing methods.
2019-05-27
ArXiv (preprint)
arxiv.org
arxiv.org
Activity-Based Analysis of Open Source Software Contributors: Roles and Dynamics
Jinghui Cheng
Jin L.C. Guo
Contributors to open source software (OSS) communities assume diverse roles to take different responsibilities. One major limitation of the … (see more)current OSS tools and platforms is that they provide a uniform user interface regardless of the activities performed by the various types of contributors. This paper serves as a non-trivial first step towards resolving this challenge by demonstrating a methodology and establishing knowledge to understand how the contributors' roles and their dynamics, reflected in the activities contributors perform, are exhibited in OSS communities. Based on an analysis of user action data from 29 GitHub projects, we extracted six activities that distinguished four Active roles and five Supporting roles of OSS contributors, as well as patterns in role changes. Through the lens of the Activity Theory, these findings provided rich design guidelines for OSS tools to support diverse contributor roles.
2019-05-26
2019 IEEE/ACM 12th International Workshop on Cooperative and Human Aspects of Software Engineering (CHASE) (published)
doi.org
arxiv.org
Singular Value Automata and Approximate Minimization
Borja Balle
Prakash Panangaden
Doina Precup
The present paper uses spectral theory of linear operators to construct approximately minimal realizations of weighted languages. Our new co… (see more)ntributions are: (i) a new algorithm for the SVD decomposition of infinite Hankel matrices based on their representation in terms of weighted automata, (ii) a new canonical form for weighted automata arising from the SVD of its corresponding Hankel matrix and (iii) an algorithm to construct approximate minimizations of given weighted automata by truncating the canonical form. We give bounds on the quality of our approximation.
2019-05-26
Mathematical Structures in Computer Science (published)
doi.org
arxiv.org