Dzmitry Bahdanau

2021-07-17

International Joint Conference on Neural Networks (published)

Combating False Negatives in Adversarial Imitation Learning

Konrad Żołna

Chitwan Saharia

Léonard Boussioux

David Yu-tung Hui

Maxime Chevalier-Boisvert

In adversarial imitation learning, a discriminator is trained to differentiate agent episodes from expert demonstrations representing the de… (see more)sired behavior. However, as the trained policy learns to be more successful, the negative examples (the ones produced by the agent) become increasingly similar to expert ones. Despite the fact that the task is successfully accomplished in some of the agent's trajectories, the discriminator is trained to output low values for them. We hypothesize that this inconsistent training signal for the discriminator can impede its learning, and consequently leads to worse overall performance of the agent. We show experimental evidence for this hypothesis and that the ‘False Negatives’ (i.e. successful agent episodes) significantly hinder adversarial imitation learning, which is the first contribution of this paper. Then, we propose a method to alleviate the impact of false negatives and test it on the BabyAI environment. This method consistently improves sample efficiency over the baselines by at least an order of magnitude.

2021-07-17

International Joint Conference on Neural Networks (published)

BabyAI 1.1

David Y. T. Hui

Maxime Chevalier-Boisvert

The BabyAI platform is designed to measure the sample efficiency of training an agent to follow grounded-language instructions. BabyAI 1.0 p… (see more)resents baseline results of an agent trained by deep imitation or reinforcement learning. BabyAI 1.1 improves the agent's architecture in three minor ways. This increases reinforcement learning sample efficiency by up to 3 times and improves imitation learning performance on the hardest level from 77 % to 90.4 %. We hope that these improvements increase the computational efficiency of BabyAI experiments and help users design better agents.

2020-12-31

(published)

HADE: Hierarchical Affective Dialog Encoder for Personality Recognition in Conversation

Nabiha Asghar

P. Poupart

Jesse Hoey

Xin Jiang

M. Bradley

P. J. Lang

Affective

Kyunghyun Cho

Bart Van Merriënboer

Caglar Gul-666

Holger Fethi Bougares

Pierre Colombo

Wojciech Witon

Ashutosh Modi

J. Kennedy

Mubbasir Kapadia

J. Devlin

Ming-Wei Chang

Kenton Lee … (see 35 more)

Jennifer Golbeck

Cristina Robles

Xiaodong Gu

Kang Min

Yoo Jung-Woo

Iulian Serban

Aaron C. Courville

Alireza Souri

Shafigheh Hosseinpour

Amir Ma-822

Edward P Tighe

Jennifer C. Ureta

B. Andrei

Ashish Vaswani

Noam Shazeer

Niki Parmar

David I. Watson

Lee Anna

Clark. 1992

On

Thomas Wolf

Lysandre Debut

Julien Victor Sanh

Clement Chaumond

Anthony Delangue

Pier-339 Moi

Tim ric Cistac

Rémi Rault

Morgan Louf

Funtowicz

Julien Chaumond

2020-12-31

(published)

www.semanticscholar.org

CLOSURE: Assessing Systematic Generalization of CLEVR Models

Harm de Vries

Timothy J. O'Donnell

Shikhar Murty

Philippe Beaudoin

Aaron Courville

The CLEVR dataset of natural-looking questions about 3D-rendered scenes has recently received much attention from the research community. A … (see more)number of models have been proposed for this task, many of which achieved very high accuracies of around 97-99%. In this work, we study how systematic the generalization of such models is, that is to which extent they are capable of handling novel combinations of known linguistic constructs. To this end, we test models' understanding of referring expressions based on matching object properties (such as e.g. "the object that is the same size as the red ball") in novel contexts. Our experiments on the thereby constructed CLOSURE benchmark show that state-of-the-art models often do not exhibit systematicity after being trained on CLEVR. Surprisingly, we find that an explicitly compositional Neural Module Network model also generalizes badly on CLOSURE, even when it has access to the ground-truth programs at test time. We improve the NMN's systematic generalization by developing a novel Vector-NMN module architecture with vector-valued inputs and outputs. Lastly, we investigate the extent to which few-shot transfer learning can help models that are pretrained on CLEVR to adapt to CLOSURE. Our few-shot learning experiments contrast the adaptation behavior of the models with intermediate discrete programs with that of the end-to-end continuous models.

2019-12-11

ArXiv (preprint)

Automated curriculum generation for Policy Gradients from Demonstrations

Anirudh Srinivasan

Maxime Chevalier-Boisvert

2019-11-30

arXiv (preprint)

BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learning

Maxime Chevalier-Boisvert

Allowing humans to interactively train artificial agents to understand language instructions is desirable for both practical and scientific … (see more)reasons, but given the poor data efficiency of the current learning methods, this goal may require substantial research efforts. Here, we introduce the BabyAI research platform to support investigations towards including humans in the loop for grounded language learning. The BabyAI platform comprises an extensible suite of 19 levels of increasing difficulty. The levels gradually lead the agent towards acquiring a combinatorially rich synthetic language which is a proper subset of English. The platform also provides a heuristic expert agent for the purpose of simulating a human teacher. We report baseline results and estimate the amount of human involvement that would be required to train a neural network-based agent on some of the BabyAI levels. We put forward strong evidence that current deep learning methods are not yet sufficiently sample efficient when it comes to learning a language with compositional properties.

2018-12-31

International Conference on Learning Representations (poster)

Systematic Generalization: What Is Required and Can It Be Learned?

Dzmitry Bahdanau*

Shikhar Murty*

2018-12-31

ICLR.cc/2019/Conference (poster)

BabyAI: First Steps Towards Grounded Language Learning With a Human In the Loop

Maxime Chevalier-Boisvert

Allowing humans to interactively train artificial agents to understand language instructions is desirable for both practical and scientific … (see more)reasons, but given the poor data efficiency of the current learning methods, this goal may require substantial research efforts. Here, we introduce the BabyAI research platform to support investigations towards including humans in the loop for grounded language learning. The BabyAI platform comprises an extensible suite of 19 levels of increasing difficulty. The levels gradually lead the agent towards acquiring a combinatorially rich synthetic language which is a proper subset of English. The platform also provides a heuristic expert agent for the purpose of simulating a human teacher. We report baseline results and estimate the amount of human involvement that would be required to train a neural network-based agent on some of the BabyAI levels. We put forward strong evidence that current deep learning methods are not yet sufficiently sample efficient when it comes to learning a language with compositional properties.

2018-10-17

arXiv.org (preprint)

dblp.uni-trier.de

Commonsense mining as knowledge base completion? A study on the impact of novelty

Stanisław Jastrzębski

Seyedarian Hosseini

Michael Noukhovitch

Jackie Chi Kit Cheung

Commonsense knowledge bases such as ConceptNet represent knowledge in the form of relational triples. Inspired by the recent work by Li et a… (see more)l., we analyse if knowledge base completion models can be used to mine commonsense knowledge from raw text. We propose novelty of predicted triples with respect to the training set as an important factor in interpreting results. We critically analyse the difficulty of mining novel commonsense knowledge, and show that a simple baseline method outperforms the previous state of the art on predicting more novel.

2017-12-31

OpenAlex (published)

An Actor-Critic Algorithm for Sequence Prediction

We present an approach to training neural networks to generate sequences using actor-critic methods from reinforcement learning (RL). Curren… (see more)t log-likelihood training methods are limited by the discrepancy between their training and testing modes, as models must generate tokens conditioned on their previous guesses rather than the ground-truth tokens. We address this problem by introducing a \textit{critic} network that is trained to predict the value of an output token, given the policy of an \textit{actor} network. This results in a training procedure that is much closer to the test phase, and allows us to directly optimize for a task-specific score such as BLEU. Crucially, since we leverage these techniques in the supervised learning setting rather than the traditional RL setting, we condition the critic network on the ground-truth output. We show that our method leads to improved performance on both a synthetic task, and for German-English machine translation. Our analysis paves the way for such methods to be applied in natural language generation tasks, such as machine translation, caption generation, and dialogue modelling.

2016-12-31

ICLR.cc/2017/conference (poster)

Learning to Compute Word Embeddings on the Fly

Tom Bosc

Stanisław Jastrzębski

Edward Grefenstette

Pascal Vincent

Words in natural language follow a Zipfian distribution whereby some words are frequent but most are rare. Learning representations for word… (see more)s in the "long tail" of this distribution requires enormous amounts of data. Representations of rare words trained directly on end tasks are usually poor, requiring us to pre-train embeddings on external data, or treat all rare words as out-of-vocabulary words with a unique representation. We provide a method for predicting embeddings of rare words on the fly from small amounts of auxiliary data with a network trained end-to-end for the downstream task. We show that this improves results against baselines where embeddings are trained on the end task for reading comprehension, recognizing textual entailment and language modeling.

2016-12-31

Computing Research Repository (published)