Publications

Dynamic Capacity Networks
Amjad Almahairi
Nicolas Ballas
Tim Cooijmans
Yin Zheng
Hugo Larochelle
Aaron Courville
We introduce the Dynamic Capacity Network (DCN), a neural network that can adaptively assign its capacity across different portions of the i… (voir plus)nput data. This is achieved by combining modules of two types: low-capacity sub-networks and high-capacity sub-networks. The low-capacity sub-networks are applied across most of the input, but also provide a guide to select a few portions of the input on which to apply the high-capacity sub-networks. The selection is made using a novel gradient-based attention mechanism, that efficiently identifies input regions for which the DCN's output is most sensitive and to which we should devote more capacity. We focus our empirical evaluation on the Cluttered MNIST and SVHN image datasets. Our findings indicate that DCNs are able to drastically reduce the number of computations, compared to traditional convolutional neural networks, while maintaining similar or even better performance.
2016-06-10
Proceedings of The 33rd International Conference on Machine Learning (publié)
doi.org
proceedings.mlr.press
First Result on Arabic Neural Machine Translation
Amjad Almahairi
Kyunghyun Cho
Nizar Habash
Aaron Courville
Neural machine translation has become a major alternative to widely used phrase-based statistical machine translation. We notice however tha… (voir plus)t much of research on neural machine translation has focused on European languages despite its language agnostic nature. In this paper, we apply neural machine translation to the task of Arabic translation (Ar En) and compare it against a standard phrase-based translation system. We run extensive comparison using various configurations in preprocessing Arabic script and show that the phrase-based and neural translation systems perform comparably to each other and that proper preprocessing of Arabic script has a similar effect on both of the systems. We however observe that the neural machine translation significantly outperform the phrase-based system on an out-of-domain test set, making it attractive for real-world deployment.
2016-06-07
ArXiv (prépublication)
arxiv.org
arxiv.org
Brain Tumor Segmentation with Deep Neural Networks
Mohammad Havaei
Axel Davy
David Warde-Farley
Antoine Biard
Aaron Courville
Yoshua Bengio
Chris Pal
Pierre-Marc Jodoin
Hugo Larochelle
2016-05-20
Medical Image Analysis (inconnu)
doi.org
arxiv.org
Movie Description
Anna Rohrbach
Atousa Torabi
Marcus Rohrbach
Niket Tandon
Christopher Pal
Hugo Larochelle
Aaron Courville
Bernt Schiele
Audio description (AD) provides linguistic descriptions of movies and allows visually impaired people to follow a movie along with their pee… (voir plus)rs. Such descriptions are by design mainly visual and thus naturally form an interesting data source for computer vision and computational linguistics. In this work we propose a novel dataset which contains transcribed ADs, which are temporally aligned to full length movies. In addition we also collected and aligned movie scripts used in prior work and compare the two sources of descriptions. We introduce the Large Scale Movie Description Challenge (LSMDC) which contains a parallel corpus of 128,118 sentences aligned to video clips from 200 movies (around 150 h of video in total). The goal of the challenge is to automatically generate descriptions for the movie clips. First we characterize the dataset by benchmarking different approaches for generating video descriptions. Comparing ADs to scripts, we find that ADs are more visual and describe precisely what is shown rather than what should happen according to the scripts created prior to movie production. Furthermore, we present and compare the results of several teams who participated in the challenges organized in the context of two workshops at ICCV 2015 and ECCV 2016.
2016-05-11
ArXiv (prépublication)
doi.org
arxiv.org
Nash equilibria in the two-player kidney exchange game
Margarida Carvalho
Andrea Lodi
João Pedro Pedroso
Ana Luiza D'ávila Viana
2016-04-20
Mathematical Programming (publié)
doi.org
Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models
Iulian V. Serban
Alessandro Sordoni
Yoshua Bengio
Aaron Courville
Joelle Pineau
We investigate the task of building open domain, conversational dialogue systems based on large dialogue corpora using generative models. Ge… (voir plus)nerative models produce system responses that are autonomously generated word-by-word, opening up the possibility for realistic, flexible interactions. In support of this goal, we extend the recently proposed hierarchical recurrent encoder-decoder neural network to the dialogue domain, and demonstrate that this model is competitive with state-of-the-art neural language models and back-off n-gram models. We investigate the limitations of this and similar approaches, and show how its performance can be improved by bootstrapping the learning from a larger question-answer pair corpus and from pretrained word embeddings.
2016-03-04
Proceedings of the AAAI Conference on Artificial Intelligence (publié)
doi.org
arxiv.org
A Controller Recognizer Framework: How necessary is recognition for control?
Marcin Moczulski
Kelvin Xu
Aaron Courville
Kyunghyun Cho
Recently there has been growing interest in building active visual object recognizers, as opposed to the usual passive recognizers which cla… (voir plus)ssifies a given static image into a predefined set of object categories. In this paper we propose to generalize these recently proposed end-to-end active visual recognizers into a controller-recognizer framework. A model in the controller-recognizer framework consists of a controller, which interfaces with an external manipulator, and a recognizer which classifies the visual input adjusted by the manipulator. We describe two most recently proposed controller-recognizer models: recurrent attention model and spatial transformer network as representative examples of controller-recognizer models. Based on this description we observe that most existing end-to-end controller-recognizers tightly, or completely, couple a controller and recognizer. We ask a question whether this tight coupling is necessary, and try to answer this empirically by building a controller-recognizer model with a decoupled controller and recognizer. Our experiments revealed that it is not always necessary to tightly couple them and that by decoupling a controller and recognizer, there is a possibility of building a generic controller that is pretrained and works together with any subsequent recognizer.
2016-02-17
ICLR.cc/2016/workshop (inconnu)
openreview.net
openreview.net
Variance Reduction in SGD by Distributed Importance Sampling
Guillaume Alain
Alex Lamb
Chinnadhurai Sankar
Aaron Courville
Yoshua Bengio
Humans are able to accelerate their learning by selecting training materials that are the most informative and at the appropriate level of d… (voir plus)ifficulty. We propose a framework for distributing deep learning in which one set of workers search for the most informative examples in parallel while a single worker updates the model on examples selected by importance sampling. This leads the model to update using an unbiased estimate of the gradient which also has minimum variance when the sampling proposal is proportional to the L2-norm of the gradient. We show experimentally that this method reduces gradient variance even in a context where the cost of synchronization across machines cannot be ignored, and where the factors for importance sampling are not updated instantly across the training set.
2016-02-16
ICLR.cc/2016/workshop (inconnu)
openreview.net
openreview.net
Fault-Tolerant Associative Memories Based on $c$-Partite Graphs
François Leduc-Primeau
Vincent Gripon
Michael G. Rabbat
Warren J. Gross
Associative memories allow the retrieval of previously stored messages given a part of their content. In this paper, we are interested in as… (voir plus)sociative memories based on c-partite graphs that were recently introduced. These memories are almost optimal in terms of the amount of storage they require (efficiency) and allow retrieving messages with low complexity. We propose a generic implementation model for the retrieval algorithm that can be readily mapped to an integrated circuit and study the retrieval performance when hardware components are affected by faults. We show using analytical and simulation results that these associative memories can be made resilient to circuit faults with a minor modification of the retrieval algorithm. In one example, the memory retains 88% of its efficiency when 1% of the storage cells are faulty, or 98% when 0.1% of the binary outputs of the retrieval algorithm are faulty. When considering storage faults, the fault tolerance exhibited by the proposed associative memory can be comparable to using a capacity-achieving error correction code for protecting the stored information.
2016-02-14
IEEE Transactions on Signal Processing (publié)
doi.org
Discriminative Regularization for Generative Models
Alex Lamb
Vincent Dumoulin
Aaron Courville
We explore the question of whether the representations learned by classifiers can be used to enhance the quality of generative models. Our c… (voir plus)onjecture is that labels correspond to characteristics of natural data which are most salient to humans: identity in faces, objects in images, and utterances in speech. We propose to take advantage of this by using the representations from discriminative classifiers to augment the objective function corresponding to a generative model. In particular we enhance the objective function of the variational autoencoder, a popular generative model, with a discriminative regularization term. We show that enhancing the objective function in this way leads to samples that are clearer and have higher visual quality than the samples from the standard variational autoencoders.
2016-02-08
ArXiv (prépublication)
arxiv.org
arxiv.org
Deep Learning Vector Quantization
Harm de Vries
Roland Memisevic
Aaron Courville
. While deep neural nets (DNN’s) achieve impressive performance on image recognition tasks, previous studies have reported that DNN’s gi… (voir plus)ve high confidence predictions for unrecognizable images. Motivated by the observation that such fooling examples might be caused by the extrapolating nature of the log-softmax, we propose to combine neural networks with Learning Vector Quantization (LVQ). Our proposed method, called Deep LVQ (DLVQ), achieves comparable performance on MNIST while being more robust against fooling and adversarial examples.
2015-12-31
The European Symposium on Artificial Neural Networks (publié)
dblp.uni-trier.de
Editorial on Special Issue on Probabilistic Models for Biomedical Image Analysis.
Tal Arbel
Manuel Jorge Cardoso
William M. Wells III
Albert C. S. Chung
Doina Precup
2015-12-31
Computer Vision and Image Understanding (publié)
doi.org