Publications

Structured Conditional Continuous Normalizing Flows for Efficient Amortized Inference in Graphical Models
Christian Dietrich Weilbach
Boyan Beronov
Frank N. Wood
William Harvey
We exploit minimally faithful inversion of graphical model structures to specify sparse continuous normalizing flows (CNFs) for amortized i… (voir plus)nference. We find that the sparsity of this factorization can be exploited to reduce the numbers of parameters in the neural network, adaptive integration steps of the flow, and consequently FLOPs at both training and inference time without decreasing performance in comparison to unconstrained flows. By expressing the structure inversion as a compilation pass in a probabilistic programming language, we are able to apply it in a novel way to models as complex as convolutional neural networks. Furthermore, we extend the training objective for CNFs in the context of inference amortization to the symmetric Kullback-Leibler divergence, and demonstrate its theoretical and practical advantages.
2020-06-02
Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics (publié)
proceedings.mlr.press
proceedings.mlr.press
Tensorized Random Projections
Beheshteh T. Rakhshan
Guillaume Rabusseau
2020-06-02
Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics (publié)
proceedings.mlr.press
proceedings.mlr.press
On the interplay between noise and curvature and its effect on optimization and generalization
Valentin Thomas
Fabian Pedregosa
Bart Van Merriënboer
Pierre-Antoine Mangazol
Yoshua Bengio
Nicolas Le Roux
The speed at which one can minimize an expected loss using stochastic methods depends on two properties: the curvature of the loss and the v… (voir plus)ariance of the gradients. While most previous works focus on one or the other of these properties, we explore how their interaction affects optimization speed. Further, as the ultimate goal is good generalization performance, we clarify how both curvature and noise are relevant to properly estimate the generalization gap. Realizing that the limitations of some existing works stems from a confusion between these matrices, we also clarify the distinction between the Fisher matrix, the Hessian, and the covariance matrix of the gradients.
2020-06-02
Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics (publié)
doi.org
proceedings.mlr.press
The Neurobiology of Social Distance
Danilo Bzdok
Robin I. M. Dunbar
2020-06-02
Trends in Cognitive Sciences (publié)
doi.org
A Tight and Unified Analysis of Gradient-Based Methods for a Whole Spectrum of Differentiable Games
Waïss Azizian
Ioannis Mitliagkas
Simon Lacoste-Julien
Gauthier Gidel
We consider differentiable games where the goal is to find a Nash equilibrium. The machine learning community has recently started using var… (voir plus)iants of the gradient method (GD). Prime examples are extragradient (EG), the optimistic gradient method (OG) and consensus optimization (CO), which enjoy linear convergence in cases like bilinear games, where the standard GD fails. The full benefits of theses relatively new methods are not known as there is no unified analysis for both strongly monotone and bilinear games. We provide new analyses of the EG's local and global convergence properties and use is to get a tighter global convergence rate for OG and CO. Our analysis covers the whole range of settings between bilinear and strongly monotone games. It reveals that these methods converge via different mechanisms at these extremes; in between, it exploits the most favorable mechanism for the given problem. We then prove that EG achieves the optimal rate for a wide class of algorithms with any number of extrapolations. Our tight analysis of EG's convergence rate in games shows that, unlike in convex minimization, EG may be much faster than GD.
2020-06-02
International Conference on Artificial Intelligence and Statistics (inconnu)
doi.org
proceedings.mlr.press
Value Preserving State-Action Abstractions
David Abel
Nathan Umbanhowar
Khimya Khetarpal
Dilip Arumugam
Doina Precup
Michael L. Littman
2020-06-02
Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics (publié)
proceedings.mlr.press
proceedings.mlr.press
The need for privacy with public digital contact tracing during the COVID-19 pandemic
Yoshua Bengio
Richard Janda
Yun William Yu
Daphne Ippolito
Max Jarvie
Dan Pilat
Brooke Struck
Sekoul Krastev
Abhinav Sharma
2020-06-01
The Lancet Digital Health (publié)
doi.org
Training End-to-End Analog Neural Networks with Equilibrium Propagation
Jack D. Kendall
Ross D. Pantone
Kalpana Manickavasagam
Yoshua Bengio
Benjamin Scellier
We introduce a principled method to train end-to-end analog neural networks by stochastic gradient descent. In these analog neural networks,… (voir plus) the weights to be adjusted are implemented by the conductances of programmable resistive devices such as memristors [Chua, 1971], and the nonlinear transfer functions (or `activation functions') are implemented by nonlinear components such as diodes. We show mathematically that a class of analog neural networks (called nonlinear resistive networks) are energy-based models: they possess an energy function as a consequence of Kirchhoff's laws governing electrical circuits. This property enables us to train them using the Equilibrium Propagation framework [Scellier and Bengio, 2017]. Our update rule for each conductance, which is local and relies solely on the voltage drop across the corresponding resistor, is shown to compute the gradient of the loss function. Our numerical simulations, which use the SPICE-based Spectre simulation framework to simulate the dynamics of electrical circuits, demonstrate training on the MNIST classification task, performing comparably or better than equivalent-size software-based neural networks. Our work can guide the development of a new generation of ultra-fast, compact and low-power neural networks supporting on-chip learning.
2020-06-01
ArXiv (prépublication)
arxiv.org
arxiv.org
Cross-layer communication over fading channels with adaptive decision feedback
Borna Sayedana
Aditya Mahajan
E. Yeh
In this paper, cross-layer design of transmitting data packets over AWGN fading channel with adaptive decision feedback is considered. The t… (voir plus)ransmitter decides the number of packets to transmit and the threshold of the decision feedback based on the queue length and the channel state. The transmit power is chosen such that the probability of error is below a pre-specified threshold. We model the system as a Markov decision process and use ideas from lattice theory to establish qualitative properties of optimal transmission strategies. In particular, we show that: (i) if the channel state remains the same and the number of packets in the queue increase, then the optimal policy either transmits more packets or uses a smaller decision feedback threshold or both; and (ii) if the number of packets in the queue remain the same and the channel quality deteriorates, then the optimal policy either transmits fewer packets or uses a larger threshold for the decision feedback or both. We also show under rate constraints that if the channel gains for all channel states are above a threshold, then the “or” in the above characterization can be replaced by “and”. Finally, we present a numerical example showing that adaptive decision feedback significantly improves the power-delay trade-off as compared with the case of no feedback.
2020-05-31
International Symposium on Modeling and Optimization in Mobile, Ad-Hoc and Wireless Networks (publié)
dblp.uni-trier.de
Multi-Image Super-Resolution for Remote Sensing using Deep Recurrent Networks
Md Rifat Arefin
Vincent Michalski
Pierre-Luc St-Charles
Alfredo Kalaitzis
Sookyung Kim
Samira E. Kahou
Yoshua Bengio
High-resolution satellite imagery is critical for various earth observation applications related to environment monitoring, geoscience, fore… (voir plus)casting, and land use analysis. However, the acquisition cost of such high-quality imagery due to the scarcity of providers and needs for high-frequency revisits restricts its accessibility in many fields. In this work, we present a data-driven, multi-image super resolution approach to alleviate these problems. Our approach is based on an end-to-end deep neural network that consists of an encoder, a fusion module, and a decoder. The encoder extracts co-registered highly efficient feature representations from low-resolution images of a scene. A Gated Re-current Unit (GRU)-based module acts as the fusion module, aggregating features into a combined representation. Finally, a decoder reconstructs the super-resolved image. The proposed model is evaluated on the PROBA-V dataset released in a recent competition held by the European Space Agency. Our results show that it performs among the top contenders and offers a new practical solution for real-world applications.
2020-05-31
2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) (publié)
doi.org
Population variability in social brain morphology for social support, household size and friendship satisfaction
Arezoo Taebi
Hannah Kiesow
Kai Vogeley
Leonhard Schilbach
Boris C. Bernhardt
Danilo Bzdok
The social brain hypothesis proposes that the complexity of human brains has coevolved with increasing complexity of social interactions in … (voir plus)primate societies. The present study explored the possible relationships between brain morphology and the richness of more intimate ‘inner’ and wider ‘outer’ social circles by integrating Bayesian hierarchical modeling with a large cohort sample from the UK Biobank resource (n = 10 000). In this way, we examined population volume effects in 36 regions of the ‘social brain’, ranging from lower sensory to higher associative cortices. We observed strong volume effects in the visual sensory network for the group of individuals with satisfying friendships. Further, the limbic network displayed several brain regions with substantial volume variations in individuals with a lack of social support. Our population neuroscience approach thus showed that distinct networks of the social brain show different patterns of volume variations linked to the examined social indices.
2020-05-31
Social Cognitive and Affective Neuroscience (publié)
doi.org
Restless bandits: indexability and computation of Whittle index
Nima Akbarzadeh
Aditya Mahajan
Restless bandits are a class of sequential resource allocation problems concerned with allocating one or more resources among several altern… (voir plus)ative processes where the evolution of the process depends on the resource allocated to them. Such models capture the fundamental trade-offs between exploration and exploitation. In 1988, Whittle developed an index heuristic for restless bandit problems which has emerged as a popular solution approach due to its simplicity and strong empirical performance. The Whittle index heuristic is applicable if the model satisfies a technical condition known as indexability. In this paper, we present two general sufficient conditions for indexability and identify simpler to verify refinements of these conditions. We then present a general algorithm to compute Whittle index for indexable restless bandits. Finally, we present a detailed numerical study which affirms the strong performance of the Whittle index heuristic.
2020-05-31
arXiv.org (prépublication)
dblp.uni-trier.de