Publications

Language Gans Falling Short
Massimo Caccia
Lucas Caccia
William Fedus
Hugo Larochelle
Joelle Pineau
Laurent Charlin
Generating high-quality text with sufficient diversity is essential for a wide range of Natural Language Generation (NLG) tasks. Maximum-Lik… (see more)elihood (MLE) models trained with teacher forcing have consistently been reported as weak baselines, where poor performance is attributed to exposure bias (Bengio et al., 2015; Ranzato et al., 2015); at inference time, the model is fed its own prediction instead of a ground-truth token, which can lead to accumulating errors and poor samples. This line of reasoning has led to an outbreak of adversarial based approaches for NLG, on the account that GANs do not suffer from exposure bias. In this work, we make several surprising observations which contradict common beliefs. First, we revisit the canonical evaluation framework for NLG, and point out fundamental flaws with quality-only evaluation: we show that one can outperform such metrics using a simple, well-known temperature parameter to artificially reduce the entropy of the model's conditional distributions. Second, we leverage the control over the quality / diversity trade-off given by this parameter to evaluate models over the whole quality-diversity spectrum and find MLE models constantly outperform the proposed GAN variants over the whole quality-diversity space. Our results have several implications: 1) The impact of exposure bias on sample quality is less severe than previously thought, 2) temperature tuning provides a better quality / diversity trade-off than adversarial training while being easier to train, easier to cross-validate, and less computationally expensive. Code to reproduce the experiments is available at github.com/pclucas14/GansFallingShort
2019-12-31
ICLR (published)
doi.org
openreview.net
Learning Classical Planning Transition Functions by Deep Neural Networks
Michaela Urbanovská
Ian G Goodfellow
Yoshua Bengio
Aaron Courville
2019-12-31
(published)
www.semanticscholar.org
Learning Graph Structure With A Finite-State Automaton Layer
Daniel D. Johnson
Hugo Larochelle
Daniel Tarlow
2019-12-31
Advances in Neural Information Processing Systems 33 (NeurIPS 2020) (published)
arxiv.org
arxiv.org
Learning Long-term Dependencies Using Cognitive Inductive Biases in Self-attention RNNs
Giancarlo Kerg
Bhargav Kanuparthi
Anirudh Goyal
Kyle Goyette
Yoshua Bengio
Guillaume Lajoie
Attention and self-attention mechanisms, inspired by cognitive processes, are now central to state-of-the-art deep learning on sequential ta… (see more)sks. However, most recent progress hinges on heuristic approaches that rely on considerable memory and computational resources that scale poorly. In this work, we propose a relevancy screening mechanism, inspired by the cognitive process of memory consolidation, that allows for a scalable use of sparse self-attention with recurrence. We use simple numerical experiments to demonstrate that this mechanism helps enable recurrent systems on generalization and transfer learning tasks. Based on our results, we propose a concrete direction of research to improve scalability and generalization of attentive recurrent networks.
2019-12-31
(published)
www.semanticscholar.org
Learning to Optimize with Recurrent Hierarchical Transformers
Abhinav Moudgil
Boris Knyazev
Guillaume Lajoie
Eugene Belilovsky
2019-12-31
International Conference on Machine Learning (published)
openreview.net
openreview.net
Learning the Arrow of Time for Problems in Reinforcement Learning
Nasim Rahaman
Steffen Wolf
Anirudh Goyal
Roman Remme
Yoshua Bengio
2019-12-31
ICLR (published)
openreview.net
openreview.net
Leveraging a Fully Differentiable Integrated Assessment Model for RL and Inference
Koen Ponse
Kai-Hendrik Cohrs
Phillip Wozny
Andrew Robert Williams
Tianyu Zhang
Erman Acar
Yoshua Bengio
Aske Plaat
Thomas M. Moerland
Pierre Gentine
Gustau Camps-Valls
2019-12-31
International Conference on Machine Learning (published)
openreview.net
openreview.net
MeDAL: Medical Abbreviation Disambiguation Dataset for Natural Language Understanding Pretraining
Zhi Wen
Xing Han Lu
Siva Reddy
2019-12-31
ClinicalNLP@EMNLP (published)
doi.org
arxiv.org
Meta Attention Networks: Meta Learning Attention To Modulate Information Between Sparsely Interacting Recurrent Modules
Kanika Madan
Nan Rosemary Ke
Anirudh Goyal
Yoshua Bengio
Decomposing knowledge into interchangeable pieces promises a generalization advantage when, at some level of representation, the learner is … (see more)likely to be faced with situations requiring novel combinations of existing pieces of knowledge or computation. We hypothesize that such a decomposition of knowledge is particularly relevant for higher levels of representation as we see this at work in human cognition and natural language in the form of systematicity or systematic generalization. To study these ideas, we propose a particular training framework in which we assume that the pieces of knowledge an agent needs, as well as its reward function are stationary and can be re-used across tasks and changes in distribution. As the learner is confronted with variations in experiences, the attention selects which modules should be adapted and the parameters of those selected modules are adapted fast, while the parameters of attention mechanisms are updated slowly as meta-parameters. We find that both the meta-learning and the modular aspects of the proposed system greatly help achieve faster learning in experiments with reinforcement learning setup involving navigation in a partially observed grid world.
2019-12-31
(published)
www.semanticscholar.org
Meta-Learning State-based Eligibility Traces for More Sample-Efficient Policy Evaluation
Mingde Zhao
Ian Porada
Sitao Luan
Xiao-Wen Chang
Doina Precup
Temporal-Difference (TD) learning is a standard and very successful reinforcement learning approach, at the core of both algorithms that lea… (see more)rn the value of a given policy, as well as algorithms which learn how to improve policies. TD-learning with eligibility traces provides a way to boost sample efficiency by temporal credit assignment, i.e. deciding which portion of a reward should be assigned to predecessor states that occurred at different previous times, controlled by a parameter
2019-12-31
International Conference on Autonomous Agents and Multiagent Systems (published)
doi.org
arxiv.org
A Meta-Transfer Objective for Learning to Disentangle Causal Mechanisms
Yoshua Bengio
Tristan Deleu
Nasim Rahaman
Nan Rosemary Ke
Sébastien Lachapelle
Olexa Bilaniuk
Anirudh Goyal
Christopher Pal
We propose to meta-learn causal structures based on how fast a learner adapts to new distributions arising from sparse distributional change… (see more)s, e.g. due to interventions, actions of agents and other sources of non-stationarities. We show that under this assumption, the correct causal structural choices lead to faster adaptation to modified distributions because the changes are concentrated in one or just a few mechanisms when the learned knowledge is modularized appropriately. This leads to sparse expected gradients and a lower effective number of degrees of freedom needing to be relearned while adapting to the change. It motivates using the speed of adaptation to a modified distribution as a meta-learning objective. We demonstrate how this can be used to determine the cause-effect relationship between two observed variables. The distributional changes do not need to correspond to standard interventions (clamping a variable), and the learner has no direct knowledge of these interventions. We show that causal structures can be parameterized via continuous variables and learned end-to-end. We then explore how these ideas could be used to also learn an encoder that would map low-level observed variables to unobserved causal variables leading to faster adaptation out-of-distribution, learning a representation space where one can satisfy the assumptions of independent mechanisms and of small and sparse changes in these mechanisms due to actions and non-stationarities.
2019-12-31
ICLR (published)
doi.org
openreview.net
Modeling Route Choice with Real-Time Information: Comparing the Recursive and Non-Recursive Approaches
Xinlian Yu
Tien Mai
Jing Ding-Mastera
Song Gao
Emma Frejinger
Transportation systems are inherently uncertain due to disruptions such as bad weather, incident and the randomness of traveler’s choices.… (see more) Real-time information allows travelers to adapt to actual traffic conditions and potentially mitigate the adverse effect of uncertainty. We study the routing policy choice problems in a stochastic time-dependent (STD) network. A routing policy is defined as a decision rule applied at the end of each link that maps the realized traffic condition to the decision on the link to take next. Two types of routing policy choice models are formulated with perfect online information (POI): recursive logit model and non-recursive logit model. In the non-recursive model, a choice set of routing policies between an origin-destination (OD) pair is generated, and a probabilistic choice is modeled at the origin, while the choice of the next link at each link is a deterministic execution of the chosen routing policy. In the recursive model, the probabilistic choice of the next link is modeled at each link, following the framework of dynamic discrete choice models. The difference between the two models results from the interplay of two sources of stochasticity, i.e., nature’s probability and choice probability. The two models are equivalent when either source of stochasticity is removed, that is, in a deterministic network (as shown in Fosgerau et al., 2013) or with deterministic choice. We use an illustrative example to explore the difference between the two models when both sources of stochasticity exist, and find that when a route has state-wise stochastic dominance over the other, the recursive model predicts more extreme choice probabilities. The relation can go either way when the two routes are non-dominated. We further compare the two models in terms of computational efficiency in estimation and prediction, and flexibility in systematic utility specification and modeling correlation.
2019-12-31
(published)
www.semanticscholar.org