Publications

Learning Classical Planning Transition Functions by Deep Neural Networks

Michaela Urbanovská

Ian G Goodfellow

Aaron Courville

2019-12-31

(publié)

Learning Graph Structure With A Finite-State Automaton Layer

Daniel D. Johnson

Hugo Larochelle

Daniel Tarlow

2019-12-31

Advances in Neural Information Processing Systems 33 (NeurIPS 2020) (publié)

arxiv.org

Learning Long-term Dependencies Using Cognitive Inductive Biases in Self-attention RNNs

Attention and self-attention mechanisms, inspired by cognitive processes, are now central to state-of-the-art deep learning on sequential ta… (voir plus)sks. However, most recent progress hinges on heuristic approaches that rely on considerable memory and computational resources that scale poorly. In this work, we propose a relevancy screening mechanism, inspired by the cognitive process of memory consolidation, that allows for a scalable use of sparse self-attention with recurrence. We use simple numerical experiments to demonstrate that this mechanism helps enable recurrent systems on generalization and transfer learning tasks. Based on our results, we propose a concrete direction of research to improve scalability and generalization of attentive recurrent networks.

2019-12-31

(publié)

Learning to Optimize with Recurrent Hierarchical Transformers

2019-12-31

International Conference on Machine Learning (publié)

Learning the Arrow of Time for Problems in Reinforcement Learning

Nasim Rahaman

Steffen Wolf

Anirudh Goyal

Roman Remme

2019-12-31

ICLR (publié)

Leveraging a Fully Differentiable Integrated Assessment Model for RL and Inference

Koen Ponse

Kai-Hendrik Cohrs

Phillip Wozny

Andrew Robert Williams

Tianyu Zhang

Erman Acar

Aske Plaat

Thomas M. Moerland

Pierre Gentine

Gustau Camps-Valls

2019-12-31

International Conference on Machine Learning (publié)

MeDAL: Medical Abbreviation Disambiguation Dataset for Natural Language Understanding Pretraining

Zhi Wen

Xing Han Lu

Siva Reddy

2019-12-31

ClinicalNLP@EMNLP (publié)

arxiv.org

Meta Attention Networks: Meta Learning Attention To Modulate Information Between Sparsely Interacting Recurrent Modules

Kanika Madan

Nan Rosemary Ke

Anirudh Goyal

Decomposing knowledge into interchangeable pieces promises a generalization advantage when, at some level of representation, the learner is … (voir plus)likely to be faced with situations requiring novel combinations of existing pieces of knowledge or computation. We hypothesize that such a decomposition of knowledge is particularly relevant for higher levels of representation as we see this at work in human cognition and natural language in the form of systematicity or systematic generalization. To study these ideas, we propose a particular training framework in which we assume that the pieces of knowledge an agent needs, as well as its reward function are stationary and can be re-used across tasks and changes in distribution. As the learner is confronted with variations in experiences, the attention selects which modules should be adapted and the parameters of those selected modules are adapted fast, while the parameters of attention mechanisms are updated slowly as meta-parameters. We ﬁnd that both the meta-learning and the modular aspects of the proposed system greatly help achieve faster learning in experiments with reinforcement learning setup involving navigation in a partially observed grid world.

2019-12-31

(publié)

Meta-Learning State-based Eligibility Traces for More Sample-Efficient Policy Evaluation

Mingde Zhao

Ian Porada

Sitao Luan

Xiao-Wen Chang

Doina Precup

Temporal-Difference (TD) learning is a standard and very successful reinforcement learning approach, at the core of both algorithms that lea… (voir plus)rn the value of a given policy, as well as algorithms which learn how to improve policies. TD-learning with eligibility traces provides a way to boost sample efficiency by temporal credit assignment, i.e. deciding which portion of a reward should be assigned to predecessor states that occurred at different previous times, controlled by a parameter

2019-12-31

International Conference on Autonomous Agents and Multiagent Systems (publié)

arxiv.org

A Meta-Transfer Objective for Learning to Disentangle Causal Mechanisms

Tristan Deleu

Nasim Rahaman

Nan Rosemary Ke

Sébastien Lachapelle

Olexa Bilaniuk

Anirudh Goyal

Christopher Pal

We propose to meta-learn causal structures based on how fast a learner adapts to new distributions arising from sparse distributional change… (voir plus)s, e.g. due to interventions, actions of agents and other sources of non-stationarities. We show that under this assumption, the correct causal structural choices lead to faster adaptation to modified distributions because the changes are concentrated in one or just a few mechanisms when the learned knowledge is modularized appropriately. This leads to sparse expected gradients and a lower effective number of degrees of freedom needing to be relearned while adapting to the change. It motivates using the speed of adaptation to a modified distribution as a meta-learning objective. We demonstrate how this can be used to determine the cause-effect relationship between two observed variables. The distributional changes do not need to correspond to standard interventions (clamping a variable), and the learner has no direct knowledge of these interventions. We show that causal structures can be parameterized via continuous variables and learned end-to-end. We then explore how these ideas could be used to also learn an encoder that would map low-level observed variables to unobserved causal variables leading to faster adaptation out-of-distribution, learning a representation space where one can satisfy the assumptions of independent mechanisms and of small and sparse changes in these mechanisms due to actions and non-stationarities.

2019-12-31

ICLR (publié)

Modeling Route Choice with Real-Time Information: Comparing the Recursive and Non-Recursive Approaches

Xinlian Yu

Tien Mai

Jing Ding-Mastera

Song Gao

Emma Frejinger

Transportation systems are inherently uncertain due to disruptions such as bad weather, incident and the randomness of traveler’s choices.… (voir plus) Real-time information allows travelers to adapt to actual traffic conditions and potentially mitigate the adverse effect of uncertainty. We study the routing policy choice problems in a stochastic time-dependent (STD) network. A routing policy is defined as a decision rule applied at the end of each link that maps the realized traffic condition to the decision on the link to take next. Two types of routing policy choice models are formulated with perfect online information (POI): recursive logit model and non-recursive logit model. In the non-recursive model, a choice set of routing policies between an origin-destination (OD) pair is generated, and a probabilistic choice is modeled at the origin, while the choice of the next link at each link is a deterministic execution of the chosen routing policy. In the recursive model, the probabilistic choice of the next link is modeled at each link, following the framework of dynamic discrete choice models. The difference between the two models results from the interplay of two sources of stochasticity, i.e., nature’s probability and choice probability. The two models are equivalent when either source of stochasticity is removed, that is, in a deterministic network (as shown in Fosgerau et al., 2013) or with deterministic choice. We use an illustrative example to explore the difference between the two models when both sources of stochasticity exist, and find that when a route has state-wise stochastic dominance over the other, the recursive model predicts more extreme choice probabilities. The relation can go either way when the two routes are non-dominated. We further compare the two models in terms of computational efficiency in estimation and prediction, and flexibility in systematic utility specification and modeling correlation.

2019-12-31

(publié)