Publications

Investigating Recurrence and Eligibility Traces in Deep Q-Networks

Jean Harb

Eligibility traces in reinforcement learning are used as a bias-variance trade-off and can often speed up training time by propagating knowl… (voir plus)edge back over time-steps in a single update. We investigate the use of eligibility traces in combination with recurrent networks in the Atari domain. We illustrate the benefits of both recurrent nets and eligibility traces in some Atari games, and highlight also the importance of the optimization used in the training.

2017-04-17

ArXiv (prépublication)

RATM: Recurrent Attentive Tracking Model

Samira Ebrahimi Kahou

Vincent Michalski

Roland Memisevic

We present an attention-based modular neural framework for computer vision. The framework uses a soft attention mechanism allowing models to… (voir plus) be trained with gradient descent. It consists of three modules: a recurrent attention module controlling where to look in an image or video frame, a feature-extraction module providing a representation of what is seen, and an objective module formalizing why the model learns its attentive behavior. The attention module allows the model to focus computation on task-related information in the input. We apply the framework to several object tracking tasks and explore various design choices. We experiment with three data sets, bouncing ball, moving digits and the real-world KTH data set. The proposed Recurrent Attentive Tracking Model performs well on all three tasks and can generalize to related but previously unseen sequences from a challenging tracking data set.

2017-04-11

cv-foundation.org/CVPR/2017/BNMW (inconnu)

A Sparse Probabilistic Model of User Preference Data

Matthew J. A. Smith

Laurent Charlin

Joelle Pineau

2017-04-10

Advances in Artificial Intelligence (publié)

Multi-Timescale, Gradient Descent, Temporal Difference Learning with Linear Options

Peeyush T. Kumar

Deliberating on large or continuous state spaces have been long standing challenges in reinforcement learning. Temporal Abstraction have som… (voir plus)ewhat made this possible, but efficiently planing using temporal abstraction still remains an issue. Moreover using spatial abstractions to learn policies for various situations at once while using temporal abstraction models is an open problem. We propose here an efficient algorithm which is convergent under linear function approximation while planning using temporally abstract actions. We show how this algorithm can be used along with randomly generated option models over multiple time scales to plan agents which need to act real time. Using these randomly generated option models over multiple time scales are shown to reduce number of decision epochs required to solve the given task, hence effectively reducing the time needed for deliberation.

2017-03-18

ArXiv (prépublication)

Nifty Assignments

Nick Parlante

Julie Zelenski

Dave Feinberg

Kunal Mishra

Josh Hug

Kevin Wayne

Michael Guerzhoy

Jackie CK Cheung

François Pitt

I suspect that students learn more from our programming assignments than from our much sweated-over lectures, with their slide transitions, … (voir plus)clip art, and joke attempts. A great assignment is deliberate about where the student hours go, concentrating the student's attention on material that is interesting and useful. The best assignments solve a problem that is topical and entertaining, providing motivation for the whole stack of work. Unfortunately, creating great programming assignments is both time consuming and error prone. The Nifty Assignments special session is all about promoting and sharing the ideas and ready-to-use materials of successful assignments.

2017-03-07

Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education (publié)

Learning-based interactive segmentation using the maximum mean cycle weight formalism

S. Nilufar

D. S. Wang

J. Girgis

C. G. Palii

D. Yang

A. Blais

M. Brand

T. J. Perkins

The maximum mean cycle weight (MMCW) segmentation framework is a graph-based alternative to approaches such as GraphCut or Markov Random Fie… (voir plus)lds. It offers time- and space-efficient computation and guaranteed optimality. However, unlike GraphCut or Markov Random Fields, MMCW does not seek to segment the entire image, but rather to find the single best object within the image, according to an objective function encoded by edge weights. Its focus on a single, best object makes MMCW attractive to interactive segmentation settings, where the user indicates which objects are to be segmented. However, a provably correct way of performing interactive segmentation using the MMCW framework has never been established. Further, the question of how to develop a good objective function based on user-provided information has never been addressed. Here, we propose a three-component objective function specifically designed for use with interactive MMCW segmentation. Two of those components, representing object boundary and object interior information, can be learned from a modest amount of user-labelled data, but in a way unique to the MMCW framework. The third component allows us to extend the MMCW framework to the situation of interactive segmentation. Specifically, we show that an appropriate weighted combination of the three components guarantees that the object produced by MMCW segmentation will enclose user-specified pixels that can be chosen interactively. The component weights can either be computed a priori based on image characteristics, or online via an adaptive reweighting scheme. We demonstrate the success of the approach on several microscope image segmentation problems.

2017-02-23

Medical Imaging (publié)

Char2Wav: End-to-End Speech Synthesis

Jose Sotelo

2017-02-16

International Conference on Learning Representations (inconnu)

Deep Nets Don't Learn Via Memorization

David Krueger

Nicolas Ballas

Stanislaw Jastrzebski

Maxinder S. Kanwal

2017-02-16

International Conference on Learning Representations (inconnu)

The Option-Critic Architecture

Pierre-Luc Bacon

Jean Harb

Temporal abstraction is key to scaling up learning and planning in reinforcement learning. While planning with temporally extended actions i… (voir plus)s well understood, creating such abstractions autonomously from data has remained challenging. We tackle this problem in the framework of options [Sutton, Precup & Singh, 1999; Precup, 2000]. We derive policy gradient theorems for options and propose a new option-critic architecture capable of learning both the internal policies and the termination conditions of options, in tandem with the policy over options, and without the need to provide any additional rewards or subgoals. Experimental results in both discrete and continuous environments showcase the flexibility and efficiency of the framework.

2017-02-12

Proceedings of the AAAI Conference on Artificial Intelligence (publié)

A Hierarchical Latent Variable Encoder-Decoder Model for Generating Dialogues

Iulian V. Serban

Sequential data often possesses a hierarchical structure with complex dependencies between subsequences, such as found between the utterance… (voir plus)s in a dialogue. In an effort to model this kind of generative process, we propose a neural network-based generative architecture, with latent stochastic variables that span a variable number of time steps. We apply the proposed model to the task of dialogue response generation and compare it with recent neural network architectures. We evaluate the model performance through automatic evaluation metrics and by carrying out a human evaluation. The experiments demonstrate that our model improves upon recently proposed models and that the latent variables facilitate the generation of long outputs and maintain the context.

2017-02-11

Proceedings of the AAAI Conference on Artificial Intelligence (publié)

Multiresolution Recurrent Neural Networks: An Application to Dialogue Response Generation

Iulian V. Serban

Tim Klinger

Gerald Tesauro

Kartik Talamadupula

Bowen Zhou

Yoshua Bengio

Aaron Courville

We introduce a new class of models called multiresolution recurrent neural networks, which explicitly model natural language generation at m… (voir plus)ultiple levels of abstraction. The models extend the sequence-to-sequence framework to generate two parallel stochastic processes: a sequence of high-level coarse tokens, and a sequence of natural language words (e.g. sentences). The coarse sequences follow a latent stochastic process with a factorial representation, which helps the models generalize to new examples. The coarse sequences can also incorporate task-specific knowledge, when available. In our experiments, the coarse sequences are extracted using automatic procedures, which are designed to capture compositional structure and semantics. These procedures enable training the multiresolution recurrent neural networks by maximizing the exact joint log-likelihood over both sequences. We apply the models to dialogue response generation in the technical support domain and compare them with several competing models. The multiresolution recurrent neural networks outperform competing models by a substantial margin, achieving state-of-the-art results according to both a human evaluation study and automatic evaluation metrics. Furthermore, experiments show the proposed models generate more fluent, relevant and goal-oriented responses.

2017-02-11

Proceedings of the AAAI Conference on Artificial Intelligence (publié)