Portrait of Aristide Baratin

Aristide Baratin

Independent visiting researcher - Samsung SAIT
Supervisor
Simon Lacoste-Julien
Research Topics
Deep Learning
Website
Google Scholar
GitHub

Publications

Implicit Regularization via Neural Feature Alignment
Aristide Baratin
Thomas George
César Laurent
R Devon Hjelm
Guillaume Lajoie
Pascal Vincent
Simon Lacoste-Julien
We approach the problem of implicit regularization in deep learning from a geometrical viewpoint. We highlight a regularization effect induc… (see more)ed by a dynamical alignment of the neural tangent features introduced by Jacot et al, along a small number of task-relevant directions. This can be interpreted as a combined mechanism of feature selection and compression. By extrapolating a new analysis of Rademacher complexity bounds for linear models, we motivate and study a heuristic complexity measure that captures this phenomenon, in terms of sequences of tangent kernel classes along optimization paths.
2019-12-31
arXiv (preprint)
doi.org
proceedings.mlr.press
On the Spectral Bias of Neural Networks
Nasim Rahaman
Aristide Baratin
Devansh Arpit
Felix Draxler
Min Lin
Fred A. Hamprecht
Yoshua Bengio
Aaron Courville
Neural networks are known to be a class of highly expressive functions able to fit even random input-output mappings with …
2018-12-31
International Conference on Machine Learning (published)
doi.org
proceedings.mlr.press
Mutual Information Neural Estimation
Mohamed Ishmael Belghazi
Aristide Baratin
Sai Rajeswar
Sherjil Ozair
Yoshua Bengio
Aaron Courville
R Devon Hjelm
We argue that the estimation of mutual information between high dimensional continuous random variables can be achieved by gradient descent … (see more)over neural networks. We present a Mutual Information Neural Estimator (MINE) that is linearly scalable in dimensionality as well as in sample size, trainable through back-prop, and strongly consistent. We present a handful of applications on which MINE can be used to minimize or maximize mutual information. We apply MINE to improve adversarially trained generative models. We also use MINE to implement Information Bottleneck, applying it to supervised classification; our results demonstrate substantial improvement in flexibility and performance in these settings.
2018-07-02
International Conference on Machine Learning (unknown)
doi.org
proceedings.mlr.press
On the Spectral Bias of Deep Neural Networks
Nasim Rahaman
Devansh Arpit
Aristide Baratin
Felix Draxler
Min Lin
Fred Hamprecht
Yoshua Bengio
Aaron Courville
It is well known that over-parametrized deep neural networks (DNNs) are an overly expressive class of functions that can memorize even rando… (see more)m data with
2018-06-21
arXiv.org (preprint)
dblp.uni-trier.de
MINE: Mutual Information Neural Estimation
Ishmael Belghazi
Sai Rajeswar
Aristide Baratin
R Devon Hjelm
Aaron Courville
This paper presents a Mutual Information Neural Estimator (MINE) that is linearly scalable in dimensionality as well as in sample size. MINE… (see more) is back-propable and we prove that it is strongly consistent. We illustrate a handful of applications in which MINE is succesfully applied to enhance the property of generative models in both unsupervised and supervised settings. We apply our framework to estimate the information bottleneck, and apply it in tasks related to supervised classification problems. Our results demonstrate substantial added flexibility and improvement in these settings.
2018-05-03
ICLR.cc/2018/Conference (unknown)
openreview.net
openreview.net
A3T: Adversarially Augmented Adversarial Training
Akram Erraqabi
Aristide Baratin
Yoshua Bengio
Simon Lacoste-Julien
Recent research showed that deep neural networks are highly sensitive to so-called adversarial perturbations, which are tiny perturbations o… (see more)f the input data purposely designed to fool a machine learning classifier. Most classification models, including deep learning models, are highly vulnerable to adversarial attacks. In this work, we investigate a procedure to improve adversarial robustness of deep neural networks through enforcing representation invariance. The idea is to train the classifier jointly with a discriminator attached to one of its hidden layer and trained to filter the adversarial noise. We perform preliminary experiments to test the viability of the approach and to compare it to other standard adversarial training methods.
2018-01-11
ArXiv (preprint)
arxiv.org
arxiv.org