Publications

A Hierarchical Latent Variable Encoder-Decoder Model for Generating Dialogues
Iulian V. Serban
Alessandro Sordoni
Ryan Lowe
Laurent Charlin
Joelle Pineau
Aaron Courville
Yoshua Bengio
Sequential data often possesses hierarchical structures with complex dependencies between sub-sequences, such as found between the utterance… (see more)s in a dialogue. To model these dependencies in a generative framework, we propose a neural network-based generative architecture, with stochastic latent variables that span a variable number of time steps. We apply the proposed model to the task of dialogue response generation and compare it with other recent neural-network architectures. We evaluate the model performance through a human evaluation study. The experiments demonstrate that our model improves upon recently proposed models and that the latent variables facilitate both the generation of meaningful, long and diverse responses and maintaining dialogue state.
2016-05-01
ArXiv (preprint)
doi.org
arxiv.org
Nash equilibria in the two-player kidney exchange game
Margarida Carvalho
Andrea Lodi
João Pedro Pedroso
Ana Luiza D'ávila Viana
2016-04-21
Mathematical Programming (published)
doi.org
Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models
Iulian V. Serban
Alessandro Sordoni
Yoshua Bengio
Aaron Courville
Joelle Pineau
We investigate the task of building open domain, conversational dialogue systems based on large dialogue corpora using generative models. Ge… (see more)nerative models produce system responses that are autonomously generated word-by-word, opening up the possibility for realistic, flexible interactions. In support of this goal, we extend the recently proposed hierarchical recurrent encoder-decoder neural network to the dialogue domain, and demonstrate that this model is competitive with state-of-the-art neural language models and back-off n-gram models. We investigate the limitations of this and similar approaches, and show how its performance can be improved by bootstrapping the learning from a larger question-answer pair corpus and from pretrained word embeddings.
2016-03-05
Proceedings of the AAAI Conference on Artificial Intelligence (published)
doi.org
arxiv.org
Task Loss Estimation for Structured Prediction
Dzmitry Bahdanau
D. Serdyuk
Philemon Brakel
Nan Rosemary Ke
Jan Chorowski
Aaron Courville
Yoshua Bengio
Fault-Tolerant Associative Memories Based on $c$-Partite Graphs
François Leduc-Primeau
Vincent Gripon
Michael Rabbat
Warren Gross
Associative memories allow the retrieval of previously stored messages given a part of their content. In this paper, we are interested in as… (see more)sociative memories based on c-partite graphs that were recently introduced. These memories are almost optimal in terms of the amount of storage they require (efficiency) and allow retrieving messages with low complexity. We propose a generic implementation model for the retrieval algorithm that can be readily mapped to an integrated circuit and study the retrieval performance when hardware components are affected by faults. We show using analytical and simulation results that these associative memories can be made resilient to circuit faults with a minor modification of the retrieval algorithm. In one example, the memory retains 88% of its efficiency when 1% of the storage cells are faulty, or 98% when 0.1% of the binary outputs of the retrieval algorithm are faulty. When considering storage faults, the fault tolerance exhibited by the proposed associative memory can be comparable to using a capacity-achieving error correction code for protecting the stored information.
2016-02-15
IEEE Transactions on Signal Processing (published)
doi.org
Fault-Tolerant Associative Memories Based on $c$-Partite Graphs
François Leduc-Primeau
Vincent Gripon
Michael Rabbat
Warren Gross
Associative memories allow the retrieval of previously stored messages given a part of their content. In this paper, we are interested in as… (see more)sociative memories based on c-partite graphs that were recently introduced. These memories are almost optimal in terms of the amount of storage they require (efficiency) and allow retrieving messages with low complexity. We propose a generic implementation model for the retrieval algorithm that can be readily mapped to an integrated circuit and study the retrieval performance when hardware components are affected by faults. We show using analytical and simulation results that these associative memories can be made resilient to circuit faults with a minor modification of the retrieval algorithm. In one example, the memory retains 88% of its efficiency when 1% of the storage cells are faulty, or 98% when 0.1% of the binary outputs of the retrieval algorithm are faulty. When considering storage faults, the fault tolerance exhibited by the proposed associative memory can be comparable to using a capacity-achieving error correction code for protecting the stored information.
2016-02-01
IEEE Transactions on Signal Processing (published)
doi.org
Former NASA chief unveils $ 100 million neural chip maker KnuEdge
C. Strasser
Dean Takahashi
Tim Klinger
Gerald Tesauro
Kartik Talamadupula
Bowen Zhou
Yoshua Bengio
Aaron Courville
Medium, Moore Data, Carly Strasser from June 07, 2016 Open access to research articles has been in the news quite a bit lately (see the SciH… (see more)ub controversy, the preprints in biology discussion, and the European Union’s recent announcement). The Data-Driven Discovery team at the Moore Foundation has also been discussing open access, particularly as it relates to the publications generated by our #MooreData researchers. Our grantee population is fairly progressive when it comes to open science, and many of the outputs that they generate are already publicly available (including proposals, software, workflows, and publications). It is therefore easy for us to imagine that they would embrace a policy that mandates open access for research articles that they produce. That said, we are always open to discussions!
Medical Computer Vision and Bayesian and Graphical Models for Biomedical Imaging
Henning Müller
B. Kelm
Tal Arbel
Weidong (Tom) Cai
M. Jorge Cardoso
Georg Langs
Bjoern Menze
Dimitris N. Metaxas
Albert A. Montillo
William Wells
Shaoting Zhang
Albert C.S. Chung
M. Jenkinson
Annemie Ribbens
2016-01-01
Lecture Notes in Computer Science (published)
doi.org
Professor Forcing: A New Algorithm for Training Recurrent Networks
Anirudh Goyal
Alex Lamb
Ying Zhang
Saizheng Zhang
Aaron Courville
Yoshua Bengio
The Teacher Forcing algorithm trains recurrent networks by supplying observed sequence values as inputs during training and using the networ… (see more)k’s own one-step-ahead predictions to do multi-step sampling. We introduce the Professor Forcing algorithm, which uses adversarial domain adaptation to encourage the dynamics of the recurrent network to be the same when training the network and when sampling from the network over multiple time steps. We apply Professor Forcing to language modeling, vocal synthesis on raw waveforms, handwriting generation, and image generation. Empirically we find that Professor Forcing acts as a regularizer, improving test likelihood on character level Penn Treebank and sequential MNIST. We also find that the model qualitatively improves samples, especially when sampling for a large number of time steps. This is supported by human evaluation of sample quality. Trade-offs between Professor Forcing and Scheduled Sampling are discussed. We produce T-SNEs showing that Professor Forcing successfully makes the dynamics of the network during training and sampling more similar.
arxiv.org
Poisson Group Testing: A Probabilistic Model for Boolean Compressed Sensing
Amin Emad
Olgica Milenkovic
We introduce a novel probabilistic group testing framework, termed Poisson group testing, in which the number of defectives follows a right-… (see more)truncated Poisson distribution. The Poisson model has a number of new applications, including dynamic testing with diminishing relative rates of defectives. We consider both nonadaptive and semi-adaptive identification methods. For nonadaptive methods, we derive a lower bound on the number of tests required to identify the defectives with a probability of error that asymptotically converges to zero; in addition, we propose test matrix constructions for which the number of tests closely matches the lower bound. For semiadaptive methods, we describe a lower bound on the expected number of tests required to identify the defectives with zero error probability. In addition, we propose a stage-wise reconstruction algorithm for which the expected number of tests is only a constant factor away from the lower bound. The methods rely only on an estimate of the average number of defectives, rather than on the individual probabilities of subjects being defective.
2015-08-15
IEEE Transactions on Signal Processing (published)
doi.org
arxiv.org
Clinical Image-Based Procedures. Translational Research in Medical Imaging
Ian J. Gerard
Marta Kersten-Oertel
Simon Drouin
Jeffery Alan Hall
Kevin Petrecca
Dante De Nigris
Tal Arbel
D. Collins
2015-01-01
Lecture Notes in Computer Science (published)
doi.org
Poisson Group Testing: A Probabilistic Model for Boolean Compressed Sensing
Amin Emad
Olgica Milenkovic
We introduce a novel probabilistic group testing framework, termed Poisson group testing, in which the number of defectives follows a right-… (see more)truncated Poisson distribution. The Poisson model has a number of new applications, including dynamic testing with diminishing relative rates of defectives. We consider both nonadaptive and semi-adaptive identification methods. For nonadaptive methods, we derive a lower bound on the number of tests required to identify the defectives with a probability of error that asymptotically converges to zero; in addition, we propose test matrix constructions for which the number of tests closely matches the lower bound. For semiadaptive methods, we describe a lower bound on the expected number of tests required to identify the defectives with zero error probability. In addition, we propose a stage-wise reconstruction algorithm for which the expected number of tests is only a constant factor away from the lower bound. The methods rely only on an estimate of the average number of defectives, rather than on the individual probabilities of subjects being defective.
2014-10-20
ArXiv (preprint)
doi.org
arxiv.org