Publications

Nash equilibria in the two-player kidney exchange game

Margarida Carvalho

Andrea Lodi

João Pedro Pedroso

Ana Luiza D'ávila Viana

2016-04-20

Mathematical Programming (publié)

doi.org

Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models

Iulian V. Serban

We investigate the task of building open domain, conversational dialogue systems based on large dialogue corpora using generative models. Ge… (voir plus)nerative models produce system responses that are autonomously generated word-by-word, opening up the possibility for realistic, flexible interactions. In support of this goal, we extend the recently proposed hierarchical recurrent encoder-decoder neural network to the dialogue domain, and demonstrate that this model is competitive with state-of-the-art neural language models and back-off n-gram models. We investigate the limitations of this and similar approaches, and show how its performance can be improved by bootstrapping the learning from a larger question-answer pair corpus and from pretrained word embeddings.

2016-03-04

Proceedings of the AAAI Conference on Artificial Intelligence (publié)

doi.org

arxiv.org

A Controller Recognizer Framework: How necessary is recognition for control?

Recently there has been growing interest in building active visual object recognizers, as opposed to the usual passive recognizers which cla… (voir plus)ssifies a given static image into a predefined set of object categories. In this paper we propose to generalize these recently proposed end-to-end active visual recognizers into a controller-recognizer framework. A model in the controller-recognizer framework consists of a controller, which interfaces with an external manipulator, and a recognizer which classifies the visual input adjusted by the manipulator. We describe two most recently proposed controller-recognizer models: recurrent attention model and spatial transformer network as representative examples of controller-recognizer models. Based on this description we observe that most existing end-to-end controller-recognizers tightly, or completely, couple a controller and recognizer. We ask a question whether this tight coupling is necessary, and try to answer this empirically by building a controller-recognizer model with a decoupled controller and recognizer. Our experiments revealed that it is not always necessary to tightly couple them and that by decoupling a controller and recognizer, there is a possibility of building a generic controller that is pretrained and works together with any subsequent recognizer.

2016-02-17

ICLR.cc/2016/workshop (inconnu)

openreview.net

Task Loss Estimation for Sequence Prediction

Nan Rosemary Ke

2016-02-17

(publié)

openreview.net

Variance Reduction in SGD by Distributed Importance Sampling

Humans are able to accelerate their learning by selecting training materials that are the most informative and at the appropriate level of d… (voir plus)ifficulty. We propose a framework for distributing deep learning in which one set of workers search for the most informative examples in parallel while a single worker updates the model on examples selected by importance sampling. This leads the model to update using an unbiased estimate of the gradient which also has minimum variance when the sampling proposal is proportional to the L2-norm of the gradient. We show experimentally that this method reduces gradient variance even in a context where the cost of synchronization across machines cannot be ignored, and where the factors for importance sampling are not updated instantly across the training set.

2016-02-16

ICLR.cc/2016/workshop (inconnu)

openreview.net

Fault-Tolerant Associative Memories Based on $c$-Partite Graphs

François Leduc-Primeau

Vincent Gripon

Michael G. Rabbat

Warren J. Gross

Associative memories allow the retrieval of previously stored messages given a part of their content. In this paper, we are interested in as… (voir plus)sociative memories based on c-partite graphs that were recently introduced. These memories are almost optimal in terms of the amount of storage they require (efficiency) and allow retrieving messages with low complexity. We propose a generic implementation model for the retrieval algorithm that can be readily mapped to an integrated circuit and study the retrieval performance when hardware components are affected by faults. We show using analytical and simulation results that these associative memories can be made resilient to circuit faults with a minor modification of the retrieval algorithm. In one example, the memory retains 88% of its efficiency when 1% of the storage cells are faulty, or 98% when 0.1% of the binary outputs of the retrieval algorithm are faulty. When considering storage faults, the fault tolerance exhibited by the proposed associative memory can be comparable to using a capacity-achieving error correction code for protecting the stored information.

2016-02-14

IEEE Transactions on Signal Processing (publié)

doi.org

Discriminative Regularization for Generative Models

Alex Lamb

Vincent Dumoulin

Aaron Courville

We explore the question of whether the representations learned by classifiers can be used to enhance the quality of generative models. Our c… (voir plus)onjecture is that labels correspond to characteristics of natural data which are most salient to humans: identity in faces, objects in images, and utterances in speech. We propose to take advantage of this by using the representations from discriminative classifiers to augment the objective function corresponding to a generative model. In particular we enhance the objective function of the variational autoencoder, a popular generative model, with a discriminative regularization term. We show that enhancing the objective function in this way leads to samples that are clearer and have higher visual quality than the samples from the standard variational autoencoders.

2016-02-08

ArXiv (prépublication)

arxiv.org

Deep Learning Vector Quantization

Harm de Vries

Roland Memisevic

Aaron Courville

. While deep neural nets (DNN’s) achieve impressive performance on image recognition tasks, previous studies have reported that DNN’s gi… (voir plus)ve high conﬁdence predictions for unrecognizable images. Motivated by the observation that such fooling examples might be caused by the extrapolating nature of the log-softmax, we propose to combine neural networks with Learning Vector Quantization (LVQ). Our proposed method, called Deep LVQ (DLVQ), achieves comparable performance on MNIST while being more robust against fooling and adversarial examples.

2015-12-31

The European Symposium on Artificial Neural Networks (publié)

dblp.uni-trier.de

Editorial on Special Issue on Probabilistic Models for Biomedical Image Analysis.

Tal Arbel

Manuel Jorge Cardoso

William M. Wells III

Albert C. S. Chung

Doina Precup

2015-12-31

Computer Vision and Image Understanding (publié)

doi.org

End-to-End Attention-Based Large Vocabulary Speech Recognition

Many of the current state-of-the-art Large Vocabulary Continuous Speech Recognition Systems (LVCSR) are hybrids of neural networks and Hidde… (voir plus)n Markov Models (HMMs). Most of these systems contain separate components that deal with the acoustic modelling, language modelling and sequence decoding. We investigate a more direct approach in which the HMM is replaced with a Recurrent Neural Network (RNN) that performs sequence prediction directly at the character level. Alignment between the input features and the desired character sequence is learned automatically by an attention mechanism built into the RNN. For each predicted character, the attention mechanism scans the input sequence and chooses relevant frames. We propose two methods to speed up this operation: limiting the scan to a subset of most promising frames and pooling over time the information contained in neighboring frames, thereby reducing source sequence length. Integrating an n-gram language model into the decoding process yields recognition accuracies similar to other HMM-free RNN-based approaches.

2015-12-31

IEEE International Conference on Acoustics, Speech and Signal Processing (publié)

doi.org

arxiv.org

Former NASA chief unveils $ 100 million neural chip maker KnuEdge

C. Strasser

Dean Takahashi

Tim Klinger

Gerald Tesauro

Kartik Talamadupula

Bowen Zhou

Yoshua Bengio

Aaron Courville

Medium, Moore Data, Carly Strasser from June 07, 2016 Open access to research articles has been in the news quite a bit lately (see the SciH… (voir plus)ub controversy, the preprints in biology discussion, and the European Union’s recent announcement). The Data-Driven Discovery team at the Moore Foundation has also been discussing open access, particularly as it relates to the publications generated by our #MooreData researchers. Our grantee population is fairly progressive when it comes to open science, and many of the outputs that they generate are already publicly available (including proposals, software, workflows, and publications). It is therefore easy for us to imagine that they would embrace a policy that mandates open access for research articles that they produce. That said, we are always open to discussions!

2015-12-31

(publié)

www.semanticscholar.org

Hierarchical Memory Networks

Gerald Tesauro

Memory networks are neural networks with an explicit memory component that can be both read and written to by the network. The memory is oft… (voir plus)en addressed in a soft way using a softmax function, making end-to-end training with backpropagation possible. However, this is not computationally scalable for applications which require the network to read from extremely large memories. On the other hand, it is well known that hard attention mechanisms based on reinforcement learning are challenging to train successfully. In this paper, we explore a form of hierarchical memory network, which can be considered as a hybrid between hard and soft attention memory networks. The memory is organized in a hierarchical structure such that reading from it is done with less computation than soft attention over a flat memory, while also being easier to train than hard attention over a flat memory. Specifically, we propose to incorporate Maximum Inner Product Search (MIPS) in the training and inference procedures for our hierarchical memory network. We explore the use of various state-of-the art approximate MIPS techniques and report results on SimpleQuestions, a challenging large scale factoid question answering task.

2015-12-31

CoRR (publié)

doi.org

arxiv.org

Communauté de pratique de Mila : L'IA agentique

Formation d'entrepreneurs scientifiques

Demandes de supervision

Publications

Communauté de pratique de Mila : L'IA agentique

Formation d'entrepreneurs scientifiques

Demandes de supervision

Mots-clés populaires:

Publications