Publications

End-to-end optimization of goal-driven and visually grounded dialogue systems
Florian Strub
Harm de Vries
Jérémie Mary
Bilal Piot
Aaron Courville
Olivier Pietquin
End-to-end design of dialogue systems has recently become a popular research topic thanks to powerful tools such as encoder-decoder architec… (see more)tures for sequence-to-sequence learning. Yet, most current approaches cast human-machine dialogue management as a supervised learning problem, aiming at predicting the next utterance of a participant given the full history of the dialogue. This vision is too simplistic to render the intrinsic planning problem inherent to dialogue as well as its grounded nature , making the context of a dialogue larger than the sole history. This is why only chitchat and question answering tasks have been addressed so far using end-to-end architectures. In this paper, we introduce a Deep Reinforcement Learning method to optimize visually grounded task-oriented dialogues , based on the policy gradient algorithm. This approach is tested on a dataset of 120k dialogues collected through Mechanical Turk and provides encouraging results at solving both the problem of generating natural dialogues and the task of discovering a specific object in a complex picture.
2017-08-18
Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence (published)
doi.org
arxiv.org
A Closer Look at Memorization in Deep Networks
Devansh Arpit
Stanisław Jastrzębski
Nicolas Ballas
David Krueger
Emmanuel Bengio
Maxinder S. Kanwal
Tegan Maharaj
Asja Fischer
Aaron Courville
Yoshua Bengio
Simon Lacoste-Julien
We examine the role of memorization in deep learning, drawing connections to capacity, generalization, and adversarial robustness. While dee… (see more)p networks are capable of memorizing noise data, our results suggest that they tend to prioritize learning simple patterns first. In our experiments, we expose qualitative differences in gradient-based optimization of deep neural networks (DNNs) on noise vs. real data. We also demonstrate that for appropriately tuned explicit regularization (e.g., dropout) we can degrade DNN training performance on noise datasets without compromising generalization on real data. Our analysis suggests that the notions of effective capacity which are dataset independent are unlikely to explain the generalization performance of deep networks when trained with gradient based methods because training data itself plays an important role in determining the degree of memorization.
2017-08-05
International Conference on Machine Learning (unknown)
doi.org
proceedings.mlr.press
Adversarial Generation of Natural Language
Sandeep Subramanian
Sai Rajeswar
Francis Dutil
Christopher Pal
Aaron Courville
Generative Adversarial Networks (GANs) have gathered a lot of attention from the computer vision community, yielding impressive results for … (see more)image generation. Advances in the adversarial generation of natural language from noise however are not commensurate with the progress made in generating images, and still lag far behind likelihood based methods. In this paper, we take a step towards generating natural language with a GAN objective alone. We introduce a simple baseline that addresses the discrete output space problem without relying on gradient estimators and show that it is able to achieve state-of-the-art results on a Chinese poem generation dataset. We present quantitative results on generating sentences from context-free and probabilistic context-free grammars, and qualitative language modeling results. A conditional version is also described that can generate sequences conditioned on sentence characteristics.
2017-07-31
Proceedings of the 2nd Workshop on Representation Learning for NLP (published)
doi.org
arxiv.org
Approximate Value Iteration with Temporally Extended Actions (Extended Abstract)
Timothy A. Mann
Shie Mannor
Doina Precup
The options framework provides a concrete way to implement and reason about temporally extended actions. Existing literature has demonstrate… (see more)d the value of planning with options empirically, but there is a lack of theoretical analysis formalizing when planning with options is more efficient than planning with primitive actions. We provide a general analysis of the convergence rate of a popular Approximate Value Iteration (AVI) algorithm called Fitted Value Iteration (FVI) with options. Our analysis reveals that longer duration options and a pessimistic estimate of the value function both lead to faster convergence. Furthermore, options can improve convergence even when they are suboptimal and sparsely distributed throughout the state space. Next we consider generating useful options for planning based on a subset of landmark states. This suggests a new algorithm, Landmarkbased AVI (LAVI), that represents the value function only at landmark states. We analyze OFVI and LAVI using the proposed landmark-based options and compare the two algorithms. Our theoretical and experimental results demonstrate that options can play an important role in AVI by decreasing approximation error and inducing fast convergence.
2017-07-31
International Joint Conference on Artificial Intelligence (published)
doi.org
Plan, Attend, Generate: Character-Level Neural Machine Translation with Planning
Caglar Gulçehre
Francis Dutil
Adam Trischler
Yoshua Bengio
We investigate the integration of a planning mechanism into an encoder-decoder architecture with attention. We develop a model that can plan… (see more) ahead when it computes alignments between the source and target sequences not only for a single time-step but for the next k time-steps as well by constructing a matrix of proposed future alignments and a commitment vector that governs whether to follow or recompute the plan. This mechanism is inspired by strategic attentive reader and writer (STRAW) model, a recent neural architecture for planning with hierarchical reinforcement learning that can also learn higher level temporal abstractions. Our proposed model is end-to-end trainable with differentiable operations. We show that our model outperforms strong baselines on character-level translation task from WMT’15 with fewer parameters and computes alignments that are qualitatively intuitive.
2017-07-31
Proceedings of the 2nd Workshop on Representation Learning for NLP (published)
doi.org
arxiv.org
Predicting Success in Goal-Driven Human-Human Dialogues
Michael Noseworthy
Jackie CK Cheung
Joelle Pineau
In goal-driven dialogue systems, success is often defined based on a structured definition of the goal. This requires that the dialogue syst… (see more)em be constrained to handle a specific class of goals and that there be a mechanism to measure success with respect to that goal. However, in many human-human dialogues the diversity of goals makes it infeasible to define success in such a way. To address this scenario, we consider the task of automatically predicting success in goal-driven human-human dialogues using only the information communicated between participants in the form of text. We build a dataset from stackoverflow.com which consists of exchanges between two users in the technical domain where ground-truth success labels are available. We then propose a turn-based hierarchical neural network model that can be used to predict success without requiring a structured goal definition. We show this model outperforms rule-based heuristics and other baselines as it is able to detect patterns over the course of a dialogue and capture notions such as gratitude.
2017-07-31
SIGDIAL Conference (published)
doi.org
A Benchmark for Endoluminal Scene Segmentation of Colonoscopy Images
David Vázquez
Jorge Bernal
F. Javier Sánchez
Gloria Fernández-Esparrach
Antonio M. López
Adriana Romero
Michal Drozdzal
Aaron Courville
Colorectal cancer (CRC) is the third cause of cancer death worldwide. Currently, the standard approach to reduce CRC-related mortality is to… (see more) perform regular screening in search for polyps and colonoscopy is the screening tool of choice. The main limitations of this screening procedure are polyp miss-rate and inability to perform visual assessment of polyp malignancy. These drawbacks can be reduced by designing Decision Support Systems (DSS) aiming to help clinicians in the different stages of the procedure by providing endoluminal scene segmentation. Thus, in this paper, we introduce an extended benchmark of colonoscopy image, with the hope of establishing a new strong benchmark for colonoscopy image analysis research. We provide new baselines on this dataset by training standard fully convolutional networks (FCN) for semantic segmentation and significantly outperforming, without any further post-processing, prior results in endoluminal scene segmentation.
2017-07-25
Journal of Healthcare Engineering (published)
doi.org
arxiv.org
Self-organized Hierarchical Softmax
Yikang Shen
Shawn Tan
Christopher Pal
Aaron Courville
We propose a new self-organizing hierarchical softmax formulation for neural-network-based language models over large vocabularies. Instead … (see more)of using a predefined hierarchical structure, our approach is capable of learning word clusters with clear syntactical and semantic meaning during the language model training process. We provide experiments on standard benchmarks for language modeling and sentence compression tasks. We find that this approach is as fast as other efficient softmax approximations, while achieving comparable or even better performance relative to similar full softmax models.
2017-07-25
ArXiv (preprint)
arxiv.org
arxiv.org
A dataset and exploration of models for understanding video data through fill-in-the-blank question-answering
Tegan Maharaj
Nicolas Ballas
Anna Rohrbach
Aaron Courville
Christopher Pal
While deep convolutional neural networks frequently approach or exceed human-level performance at benchmark tasks involving static images, e… (see more)xtending this success to moving images is not straightforward. Having models which can learn to understand video is of interest for many applications, including content recommendation, prediction, summarization, event/object detection and understanding human visual perception, but many domains lack sufficient data to explore and perfect video models. In order to address the need for a simple, quantitative benchmark for developing and understanding video, we present MovieFIB, a fill-in-the-blank question-answering dataset with over 300,000 examples, based on descriptive video annotations for the visually impaired. In addition to presenting statistics and a description of the dataset, we perform a detailed analysis of 5 different models' predictions, and compare these with human performance. We investigate the relative importance of language, static (2D) visual features, and moving (3D) visual features; the effects of increasing dataset size, the number of frames sampled; and of vocabulary size. We illustrate that: this task is not solvable by a language model alone; our model combining 2D and 3D visual information indeed provides the best result; all models perform significantly worse than human-level. We provide human evaluations for responses given by different models and find that accuracy on the MovieFIB evaluation corresponds well with human judgement. We suggest avenues for improving video models, and hope that the proposed dataset can be useful for measuring and encouraging progress in this very interesting field.
2017-07-20
2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (published)
doi.org
arxiv.org
GuessWhat?! Visual Object Discovery through Multi-modal Dialogue
Harm de Vries
Florian Strub
A. Chandar
Olivier Pietquin
Hugo Larochelle
Aaron Courville
We introduce GuessWhat?!, a two-player guessing game as a testbed for research on the interplay of computer vision and dialogue systems. The… (see more) goal of the game is to locate an unknown object in a rich image scene by asking a sequence of questions. Higher-level image understanding, like spatial reasoning and language grounding, is required to solve the proposed task. Our key contribution is the collection of a large-scale dataset consisting of 150K human-played games with a total of 800K visual question-answer pairs on 66K images. We explain our design decisions in collecting the dataset and introduce the oracle and questioner tasks that are associated with the two players of the game. We prototyped deep learning models to establish initial baselines of the introduced tasks.
2017-07-20
2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (published)
doi.org
arxiv.org
Prediction of Extubation readiness in extremely preterm infants by the automated analysis of cardiorespiratory behavior: study protocol
Wissam Shalish
Lara J. Kanbar
Smita Rao
Carlos A. Robles-Rubio
Lajos Kovacs
Sanjay Chawla
Martin Keszler
Doina Precup
Karen Brown
Robert E. Kearney
Guilherme M. Sant’Anna
BackgroundExtremely preterm infants (≤ 28 weeks gestation) commonly require endotracheal intubation and mechanical ventilation (MV) to ma… (see more)intain adequate oxygenation and gas exchange. Given that MV is independently associated with important adverse outcomes, efforts should be made to limit its duration. However, current methods for determining extubation readiness are inaccurate and a significant number of infants fail extubation and require reintubation, an intervention that may be associated with increased morbidities. A variety of objective measures have been proposed to better define the optimal time for extubation, but none have proven clinically useful. In a pilot study, investigators from this group have shown promising results from sophisticated, automated analyses of cardiorespiratory signals as a predictor of extubation readiness. The aim of this study is to develop an automated predictor of extubation readiness using a combination of clinical tools along with novel and automated measures of cardiorespiratory behavior, to assist clinicians in determining when extremely preterm infants are ready for extubation.MethodsIn this prospective, multicenter observational study, cardiorespiratory signals will be recorded from 250 eligible extremely preterm infants with birth weights ≤1250 g immediately prior to their first planned extubation. Automated signal analysis algorithms will compute a variety of metrics for each infant, and machine learning methods will then be used to find the optimal combination of these metrics together with clinical variables that provide the best overall prediction of extubation readiness. Using these results, investigators will develop an Automated system for Prediction of EXtubation (APEX) readiness that will integrate the software for data acquisition, signal analysis, and outcome prediction into a single application suitable for use by medical personnel in the neonatal intensive care unit. The performance of APEX will later be prospectively validated in 50 additional infants.DiscussionThe results of this research will provide the quantitative evidence needed to assist clinicians in determining when to extubate a preterm infant with the highest probability of success, and could produce significant improvements in extubation outcomes in this population.Trial registrationClinicaltrials.gov identifier: NCT01909947. Registered on July 17 2013.Trial sponsor: Canadian Institutes of Health Research (CIHR).
2017-07-16
BMC Pediatrics (published)
doi.org
Unimodal Probability Distributions for Deep Ordinal Classification
Christopher Beckham
Christopher Pal
Probability distributions produced by the cross-entropy loss for ordinal classification problems can possess undesired properties. We propos… (see more)e a straightforward technique to constrain discrete ordinal probability distributions to be unimodal via a combination of the Poisson probability mass function and the softmax nonlinearity. We evaluate this approach on two large ordinal image datasets and obtain promising results.
2017-07-16
Proceedings of the 34th International Conference on Machine Learning (published)
proceedings.mlr.press
proceedings.mlr.press