Emmanuel Bengio

Membre industriel associé

Scientifique en apprentissage automatique, Recursion

Sujets de recherche

Apprentissage par renforcement

Apprentissage profond

GFlowNets

Modèles génératifs

Modélisation moléculaire

Biographie

Emmanuel Bengio est chercheur en ML à Valence Labs/Recursion, où il travaille sur l'intersection des GFlowNets et de la découverte de médicaments. Il a fait son doctorat sous la direction de Joelle Pineau et Doina Precup à McGill/Mila - Institut québécois d'intelligence artificielle, en se concentrant sur la compréhension de la généralisation dans la RL profonde.

Publications

A Closer Look at Memorization in Deep Networks

Devansh Arpit

Stanisław Jastrzębski

David Krueger

Maxinder S. Kanwal

We examine the role of memorization in deep learning, drawing connections to capacity, generalization, and adversarial robustness. While dee… (voir plus)p networks are capable of memorizing noise data, our results suggest that they tend to prioritize learning simple patterns first. In our experiments, we expose qualitative differences in gradient-based optimization of deep neural networks (DNNs) on noise vs. real data. We also demonstrate that for appropriately tuned explicit regularization (e.g., dropout) we can degrade DNN training performance on noise datasets without compromising generalization on real data. Our analysis suggests that the notions of effective capacity which are dataset independent are unlikely to explain the generalization performance of deep networks when trained with gradient based methods because training data itself plays an important role in determining the degree of memorization.

2017-07-16

Proceedings of the 34th International Conference on Machine Learning (publié)

doi.org

proceedings.mlr.press

Deep Nets Don't Learn Via Memorization

David Krueger

Nicolas Ballas

Stanislaw Jastrzebski

Maxinder S. Kanwal

2017-02-16

International Conference on Learning Representations (inconnu)

openreview.net

Independently Controllable Factors

Valentin Thomas

Jules Pondard

Emmanuel Bengio

Marc Sarfati

Philippe Beaudoin

Marie-Jean Meurs

Joelle Pineau

Doina Precup

Yoshua Bengio

It has been postulated that a good representation is one that disentangles the underlying explanatory factors of variation. However, it rema… (voir plus)ins an open question what kind of training framework could potentially achieve that. Whereas most previous work focuses on the static setting (e.g., with images), we postulate that some of the causal factors could be discovered if the learner is allowed to interact with its environment. The agent can experiment with different actions and observe their effects. More specifically, we hypothesize that some of these factors correspond to aspects of the environment which are independently controllable, i.e., that there exists a policy and a learnable feature for each such aspect of the environment, such that this policy can yield changes in that feature with minimal changes to other features that explain the statistical variations in the observed data. We propose a specific objective function to find such factors and verify experimentally that it can indeed disentangle independently controllable aspects of the environment without any extrinsic reward signal.

2016-12-31

arXiv (prépublication)

doi.org

arxiv.org

Independently Controllable Features

Valentin Thomas

Philippe Beaudoin

Y. Bengio