Publications

GAIT: A Geometric Approach to Information Theory
Jose Gallego-Posada
Ankit Vani
Max Schwarzer
Simon Lacoste-Julien
We advocate the use of a notion of entropy that reflects the relative abundances of the symbols in an alphabet, as well as the similarities … (see more)between them. This concept was originally introduced in theoretical ecology to study the diversity of ecosystems. Based on this notion of entropy, we introduce geometry-aware counterparts for several concepts and theorems in information theory. Notably, our proposed divergence exhibits performance on par with state-of-the-art methods based on the Wasserstein distance, but enjoys a closed-form expression that can be computed efficiently. We demonstrate the versatility of our method via experiments on a broad range of domains: training generative models, computing image barycenters, approximating empirical measures and counting modes.
2020-06-02
Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics (published)
proceedings.mlr.press
proceedings.mlr.press
Old Dog Learns New Tricks: Randomized UCB for Bandit Problems
Sharan Vaswani
Abbas Mehrabian
Audrey Durand
Branislav Kveton
2020-06-02
Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics (published)
proceedings.mlr.press
proceedings.mlr.press
A principled approach for generating adversarial images under non-smooth dissimilarity metrics
Aram-Alexandre Pooladian
Chris Finlay
Tim Hoheisel
Adam Oberman
2020-06-02
Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics (published)
proceedings.mlr.press
proceedings.mlr.press
A Reduction from Reinforcement Learning to No-Regret Online Learning
Ching-An Cheng
Remi Tachet des Combes
Byron Boots
Geoff Gordon
We present a reduction from reinforcement learning (RL) to no-regret online learning based on the saddle-point formulation of RL, by which "… (see more)any" online algorithm with sublinear regret can generate policies with provable performance guarantees. This new perspective decouples the RL problem into two parts: regret minimization and function approximation. The first part admits a standard online-learning analysis, and the second part can be quantified independently of the learning algorithm. Therefore, the proposed reduction can be used as a tool to systematically design new RL algorithms. We demonstrate this idea by devising a simple RL algorithm based on mirror descent and the generative-model oracle. For any
2020-06-02
Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics (published)
proceedings.mlr.press
proceedings.mlr.press
Stochastic Neural Network with Kronecker Flow
Chin-wei Huang
Ahmed Touati
Pascal Vincent
Gintare Karolina Dziugaite
Alexandre Lacoste
Aaron Courville
Recent advances in variational inference enable the modelling of highly structured joint distributions, but are limited in their capacity to… (see more) scale to the high-dimensional setting of stochastic neural networks. This limitation motivates a need for scalable parameterizations of the noise generation process, in a manner that adequately captures the dependencies among the various parameters. In this work, we address this need and present the Kronecker Flow, a generalization of the Kronecker product to invertible mappings designed for stochastic neural networks. We apply our method to variational Bayesian neural networks on predictive tasks, PAC-Bayes generalization bound estimation, and approximate Thompson sampling in contextual bandits. In all setups, our methods prove to be competitive with existing methods and better than the baselines.
2020-06-02
Artificial Intelligence and Statistics (published)
doi.org
proceedings.mlr.press
Structured Conditional Continuous Normalizing Flows for Efficient Amortized Inference in Graphical Models
Christian Dietrich Weilbach
Boyan Beronov
Frank N. Wood
William Harvey
We exploit minimally faithful inversion of graphical model structures to specify sparse continuous normalizing flows (CNFs) for amortized i… (see more)nference. We find that the sparsity of this factorization can be exploited to reduce the numbers of parameters in the neural network, adaptive integration steps of the flow, and consequently FLOPs at both training and inference time without decreasing performance in comparison to unconstrained flows. By expressing the structure inversion as a compilation pass in a probabilistic programming language, we are able to apply it in a novel way to models as complex as convolutional neural networks. Furthermore, we extend the training objective for CNFs in the context of inference amortization to the symmetric Kullback-Leibler divergence, and demonstrate its theoretical and practical advantages.
2020-06-02
Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics (published)
proceedings.mlr.press
proceedings.mlr.press
Tensorized Random Projections
Beheshteh T. Rakhshan
Guillaume Rabusseau
2020-06-02
Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics (published)
proceedings.mlr.press
proceedings.mlr.press
On the interplay between noise and curvature and its effect on optimization and generalization
Valentin Thomas
Fabian Pedregosa
Bart Van Merriënboer
Pierre-Antoine Mangazol
Yoshua Bengio
Nicolas Le Roux
The speed at which one can minimize an expected loss using stochastic methods depends on two properties: the curvature of the loss and the v… (see more)ariance of the gradients. While most previous works focus on one or the other of these properties, we explore how their interaction affects optimization speed. Further, as the ultimate goal is good generalization performance, we clarify how both curvature and noise are relevant to properly estimate the generalization gap. Realizing that the limitations of some existing works stems from a confusion between these matrices, we also clarify the distinction between the Fisher matrix, the Hessian, and the covariance matrix of the gradients.
2020-06-02
Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics (published)
doi.org
proceedings.mlr.press
The Neurobiology of Social Distance
Danilo Bzdok
Robin I. M. Dunbar
2020-06-02
Trends in Cognitive Sciences (published)
doi.org
A Tight and Unified Analysis of Gradient-Based Methods for a Whole Spectrum of Differentiable Games
Waïss Azizian
Ioannis Mitliagkas
Simon Lacoste-Julien
Gauthier Gidel
We consider differentiable games where the goal is to find a Nash equilibrium. The machine learning community has recently started using var… (see more)iants of the gradient method (GD). Prime examples are extragradient (EG), the optimistic gradient method (OG) and consensus optimization (CO), which enjoy linear convergence in cases like bilinear games, where the standard GD fails. The full benefits of theses relatively new methods are not known as there is no unified analysis for both strongly monotone and bilinear games. We provide new analyses of the EG's local and global convergence properties and use is to get a tighter global convergence rate for OG and CO. Our analysis covers the whole range of settings between bilinear and strongly monotone games. It reveals that these methods converge via different mechanisms at these extremes; in between, it exploits the most favorable mechanism for the given problem. We then prove that EG achieves the optimal rate for a wide class of algorithms with any number of extrapolations. Our tight analysis of EG's convergence rate in games shows that, unlike in convex minimization, EG may be much faster than GD.
2020-06-02
International Conference on Artificial Intelligence and Statistics (unknown)
doi.org
proceedings.mlr.press
Value Preserving State-Action Abstractions
David Abel
Nathan Umbanhowar
Khimya Khetarpal
Dilip Arumugam
Doina Precup
Michael L. Littman
2020-06-02
Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics (published)
proceedings.mlr.press
proceedings.mlr.press
The need for privacy with public digital contact tracing during the COVID-19 pandemic
Yoshua Bengio
Richard Janda
Yun William Yu
Daphne Ippolito
Max Jarvie
Dan Pilat
Brooke Struck
Sekoul Krastev
Abhinav Sharma
2020-06-01
The Lancet Digital Health (published)
doi.org