(Rex) Devon Hjelm

Layla El Asri

Samira Ebrahimi Kahou

Graham W. Taylor

Conditional text-to-image generation approaches commonly focus on generating a single image in a single step. One practical extension beyond… (see more) one-step generation is an interactive system that generates an image iteratively, conditioned on ongoing linguistic input / feedback. This is significantly more challenging as such a system must understand and keep track of the ongoing context and history. In this work, we present a recurrent image generation model which takes into account both the generated output up to the current step as well as all past instructions for generation. We show that our model is able to generate the background, add new objects, apply simple transformations to existing objects, and correct previous mistakes. We believe our approach is an important step toward interactive generation.

2018-11-24

arXiv.org (preprint)

dblp.uni-trier.de

Deep Graph Infomax

Petar Veličković

William Fedus

William L. Hamilton

Pietro Lio

2018-09-27

ArXiv (preprint)

Deep Graph Infomax

Petar Veličković

William Fedus

William L. Hamilton

Pietro Lio

2018-09-27

ArXiv (preprint)

Mutual Information Neural Estimation

Ishmael Belghazi

Aristide Baratin

Sai Rajeswar

Sherjil Ozair

Aaron Courville

We argue that the estimation of mutual information between high dimensional continuous random variables can be achieved by gradient descent … (see more)over neural networks. We present a Mutual Information Neural Estimator (MINE) that is linearly scalable in dimensionality as well as in sample size, trainable through back-prop, and strongly consistent. We present a handful of applications on which MINE can be used to minimize or maximize mutual information. We apply MINE to improve adversarially trained generative models. We also use MINE to implement Information Bottleneck, applying it to supervised classification; our results demonstrate substantial improvement in flexibility and performance in these settings.

2018-07-03

Proceedings of the 35th International Conference on Machine Learning (published)

proceedings.mlr.press

MINE: Mutual Information Neural Estimation

Ishmael Belghazi

Sai Rajeswar

Aristide Baratin

Aaron Courville

This paper presents a Mutual Information Neural Estimator (MINE) that is linearly scalable in dimensionality as well as in sample size. MINE… (see more) is back-propable and we prove that it is strongly consistent. We illustrate a handful of applications in which MINE is succesfully applied to enhance the property of generative models in both unsupervised and supervised settings. We apply our framework to estimate the information bottleneck, and apply it in tasks related to supervised classification problems. Our results demonstrate substantial added flexibility and improvement in these settings.

2018-05-04

ICLR.cc/2018/Conference (unknown)

openreview.net

Boundary Seeking GANs

Athul Jacob

Adam Trischler

Gerry Che

Kyunghyun Cho

Generative adversarial networks are a learning framework that rely on training a discriminator to estimate a measure of difference between a… (see more) target and generated distributions. GANs, as normally formulated, rely on the generated samples being completely differentiable w.r.t. the generative parameters, and thus do not work for discrete data. We introduce a method for training GANs with discrete data that uses the estimated difference measure from the discriminator to compute importance weights for generated samples, thus providing a policy gradient for training the generator. The importance weights have a strong connection to the decision boundary of the discriminator, and we call our method boundary-seeking GANs (BGANs). We demonstrate the effectiveness of the proposed algorithm with discrete image and character-based natural language generation. In addition, the boundary-seeking objective extends to continuous data, which can be used to improve stability of training, and we demonstrate this on Celeba, Large-scale Scene Understanding (LSUN) bedrooms, and Imagenet without conditioning.

2018-02-15

International Conference on Learning Representations (published)

dblp.uni-trier.de

Boundary Seeking GANs

Athul Jacob

Adam Trischler

Gerry Che

Kyunghyun Cho

2018-02-15

International Conference on Learning Representations (published)

dblp.uni-trier.de

Learning Generative Models with Locally Disentangled Latent Factors

Brady Neal

Alex Lamb

Sherjil Ozair

One of the most successful techniques in generative models has been decomposing a complicated generation task into a series of simpler gener… (see more)ation tasks. For example, generating an image at a low resolution and then learning to refine that into a high resolution image often improves results substantially. Here we explore a novel strategy for decomposing generation for complicated objects in which we first generate latent variables which describe a subset of the observed variables, and then map from these latent variables to the observed space. We show that this allows us to achieve decoupled training of complicated generative models and present both theoretical and experimental results supporting the benefit of such an approach.

2018-02-15

(published)

openreview.net

Boundary Seeking GANs

Athul Jacob

Adam Trischler

Gerry Che

Kyunghyun Cho

Generative adversarial networks are a learning framework that rely on training a discriminator to estimate a measure of difference between a… (see more) target and generated distributions. GANs, as normally formulated, rely on the generated samples being completely differentiable w.r.t. the generative parameters, and thus do not work for discrete data. We introduce a method for training GANs with discrete data that uses the estimated difference measure from the discriminator to compute importance weights for generated samples, thus providing a policy gradient for training the generator. The importance weights have a strong connection to the decision boundary of the discriminator, and we call our method boundary-seeking GANs (BGANs). We demonstrate the effectiveness of the proposed algorithm with discrete image and character-based natural language generation. In addition, the boundary-seeking objective extends to continuous data, which can be used to improve stability of training, and we demonstrate this on Celeba, Large-scale Scene Understanding (LSUN) bedrooms, and Imagenet without conditioning.

2018-01-01

ICLR.cc/2018/Conference (poster)

openreview.net

ACtuAL: Actor-Critic Under Adversarial Learning

Anirudh Goyal

Nan Rosemary Ke

Alex Lamb

Generative Adversarial Networks (GANs) are a powerful framework for deep generative modeling. Posed as a two-player minimax problem, GANs ar… (see more)e typically trained end-to-end on real-valued data and can be used to train a generator of high-dimensional and realistic images. However, a major limitation of GANs is that training relies on passing gradients from the discriminator through the generator via back-propagation. This makes it fundamentally difficult to train GANs with discrete data, as generation in this case typically involves a non-differentiable function. These difficulties extend to the reinforcement learning setting when the action space is composed of discrete decisions. We address these issues by reframing the GAN framework so that the generator is no longer trained using gradients through the discriminator, but is instead trained using a learned critic in the actor-critic framework with a Temporal Difference (TD) objective. This is a natural fit for sequence modeling and we use it to achieve improvements on language modeling tasks over the standard Teacher-Forcing methods.

2017-11-13

ArXiv (preprint)

GibbsNet: Iterative Adversarial Inference for Deep Graphical Models

Alex Lamb

Yaroslav Ganin

Joseph Paul Cohen

Aaron Courville

Directed latent variable models that formulate the joint distribution as …