Publications

Online Hyper-Parameter Optimization
Damien Vincent
Sylvain Gelly
Nicolas Roux
Olivier Bousquet
2018-02-14
(publié)
openreview.net
openreview.net
Sequential Coordination of Deep Models for Learning Visual Arithmetic
Eric Crawford
Guillaume Rabusseau
Joelle Pineau
Achieving machine intelligence requires a smooth integration of perception and reasoning, yet models developed to date tend to specialize in… (voir plus) one or the other; sophisticated manipulation of symbols acquired from rich perceptual spaces has so far proved elusive. Consider a visual arithmetic task, where the goal is to carry out simple arithmetical algorithms on digits presented under natural conditions (e.g. hand-written, placed randomly). We propose a two-tiered architecture for tackling this problem. The lower tier consists of a heterogeneous collection of information processing modules, which can include pre-trained deep neural networks for locating and extracting characters from the image, as well as modules performing symbolic transformations on the representations extracted by perception. The higher tier consists of a controller, trained using reinforcement learning, which coordinates the modules in order to solve the high-level task. For instance, the controller may learn in what contexts to execute the perceptual networks and what symbolic transformations to apply to their outputs. The resulting model is able to solve a variety of tasks in the visual arithmetic domain, and has several advantages over standard, architecturally homogeneous feedforward networks including improved sample efficiency.
2018-02-14
ArXiv (prépublication)
openreview.net
openreview.net
Finding Flatter Minima with SGD
Stanisław Jastrzębski
Zac Kenton
Devansh Arpit
Nicolas Ballas
Asja Fischer
Yoshua Bengio
Amos Storkey
2018-02-11
International Conference on Learning Representations (publié)
dblp.uni-trier.de
Graph Priors for Deep Neural Networks
Francis Dutil
Joseph Paul Cohen
Martin Weiss
Georgy Derevyanko
Yoshua Bengio
In this work we explore how gene-gene interaction graphs can be used as a prior for the representation of a model to construct features base… (voir plus)d on known interactions between genes. Most existing machine learning work on graphs focuses on building models when data is confined to a graph structure. In this work we focus on using the information from a graph to build better representations in our models. We use the percolate task, determining if a path exists across a grid for a set of node values, as a proxy for gene pathways. We create variants of the percolate task to explore where existing methods fail. We test the limits of existing methods in order to determine what can be improved when applying these methods to a real task. This leads us to propose new methods based on Graph Convolutional Networks (GCN) that use pooling and dropout to deal with noise in the graph prior.
2018-02-11
(publié)
openreview.net
openreview.net
Inferring Identity Factors for Grouped Examples
Shawn Tan
Christopher Pal
Aaron Courville
We propose a method for modelling groups of face images from the same identity. The model is trained to infer a distribution over the latent… (voir plus) space for identity given a small set of “training data”. One can then sample images using that latent representation to produce images of the same identity. We demonstrate that the model extracts disentangled factors for identity factors and image-specific vectors. We also perform generative classification over identities to assess its feasibility for few-shot face recognition.
2018-02-11
(publié)
openreview.net
openreview.net
Online variance-reducing optimization
Nicolas Roux
Reza Babanezhad Harikandeh
Reza Babanezhad
Pierre-Antoine Manzagol
2018-02-11
International Conference on Learning Representations (publié)
openreview.net
openreview.net
SGD S MOOTHS THE S HARPEST D IRECTIONS
Stanisław Jastrzębski
Zac Kenton
Nicolas Ballas
Asja Fischer
Yoshua Bengio
Amos Storkey
Stochastic gradient descent (SGD) is able to find regions that generalize well, even in drastically over-parametrized models such as deep ne… (voir plus)ural networks. We observe that noise in SGD controls the spectral norm and conditioning of the Hessian throughout the training. We hypothesize the cause of this phenomenon is due to the dynamics of neurons saturating their non-linearity along the largest curvature directions, thus leading to improved conditioning.
2018-02-11
(publié)
openreview.net
openreview.net
Extending the Framework of Equilibrium Propagation to General Dynamics
Benjamin Scellier
Anirudh Goyal
Jonathan Binas
Thomas Mesnard
Yoshua Bengio
2018-02-10
International Conference on Learning Representations (publié)
openreview.net
openreview.net
Learning Robust Options
Daniel J. Mankowitz
Timothy A. Mann
Pierre-Luc Bacon
Doina Precup
Shie Mannor
Robust reinforcement learning aims to produce policies that have strong guarantees even in the face of environments/transition models whose … (voir plus)parameters have strong uncertainty. Existing work uses value-based methods and the usual primitive action setting. In this paper, we propose robust methods for learning temporally abstract actions, in the framework of options. We present a Robust Options Policy Iteration (ROPI) algorithm with convergence guarantees, which learns options that are robust to model uncertainty. We utilize ROPI to learn robust options with the Robust Options Deep Q Network (RO-DQN) that solves multiple tasks and mitigates model misspecification due to model uncertainty. We present experimental results which suggest that policy iteration with linear features may have an inherent form of robustness when using coarse feature representations. In addition, we present experimental results which demonstrate that robustness helps policy iteration implemented on top of deep neural networks to generalize over a much broader range of dynamics than non-robust policy iteration.
2018-02-08
ArXiv (prépublication)
doi.org
arxiv.org
Hierarchical Adversarially Learned Inference
Ishmael Belghazi
Sai Rajeswar
Olivier Mastropietro
Negar Rostamzadeh
Jovana Mitrovic
Aaron Courville
We propose a novel hierarchical generative model with a simple Markovian structure and a corresponding inference model. Both the generative … (voir plus)and inference model are trained using the adversarial learning paradigm. We demonstrate that the hierarchical structure supports the learning of progressively more abstract representations as well as providing semantically meaningful reconstructions with different levels of fidelity. Furthermore, we show that minimizing the Jensen-Shanon divergence between the generative and inference network is enough to minimize the reconstruction error. The resulting semantically meaningful hierarchical latent structure discovery is exemplified on the CelebA dataset. There, we show that the features learned by our model in an unsupervised way outperform the best handcrafted features. Furthermore, the extracted features remain competitive when compared to several recent deep supervised approaches on an attribute prediction task on CelebA. Finally, we leverage the model's inference network to achieve state-of-the-art performance on a semi-supervised variant of the MNIST digit classification task.
2018-02-03
ArXiv (prépublication)
openreview.net
openreview.net
Patterns of reintubation in extremely preterm infants: a longitudinal cohort study
Wissam Shalish
Lara Kanbar
Martin Keszler
Sanjay Chawla
Lajos Kovacs
Smita Rao
Bogdan A Panaitescu
Alyse Laliberte
Doina Precup
Karen Brown
Robert E Kearney
Guilherme M Sant'Anna
2018-01-30
Pediatric Research (publié)
doi.org
Combining intraoperative ultrasound brain shift correction and augmented reality visualizations: a pilot study of eight cases
Ian J. Gerard
Marta Kersten-Oertel
Simon Drouin
Jeffery A. Hall
Kevin Petrecca
Dante De Nigris
Daniel A. Di Giovanni
Tal Arbel
D. Louis Collins
2018-01-25
Journal of Medical Imaging (publié)
doi.org