Portrait of Tristan Deleu

Tristan Deleu

Collaborating Alumni - Université de Montréal
Supervisor
Yoshua Bengio
Research Topics
Causality
Deep Learning
Probabilistic Models
Reinforcement Learning
Website
Google Scholar
GitHub

Publications

GFlowNet Foundations
Yoshua Bengio
Tristan Deleu
Edward J Hu
Salem Lahlou
Mo Tiwari
Emmanuel Bengio
2021-11-17
ArXiv (preprint)
arxiv.org
arxiv.org
GFlowNet Foundations
Yoshua Bengio
Tristan Deleu
Edward J Hu
Salem Lahlou
Mo Tiwari
Emmanuel Bengio
2021-11-17
ArXiv (preprint)
arxiv.org
arxiv.org
GFlowNet Foundations
Yoshua Bengio
Tristan Deleu
Edward J Hu
Salem Lahlou
Mo Tiwari
Emmanuel Bengio
2021-11-17
ArXiv (preprint)
arxiv.org
arxiv.org
GFlowNet Foundations
Yoshua Bengio
Tristan Deleu
Edward J Hu
Salem Lahlou
Mo Tiwari
Emmanuel Bengio
Generative Flow Networks (GFlowNets) have been introduced as a method to sample a diverse set of candidates in an active learning context, w… (see more)ith a training objective that makes them approximately sample in proportion to a given reward function. In this paper, we show a number of additional theoretical properties of GFlowNets. They can be used to estimate joint probability distributions and the corresponding marginal distributions where some variables are unspecified and, of particular interest, can represent distributions over composite objects like sets and graphs. GFlowNets amortize the work typically done by computationally expensive MCMC methods in a single but trained generative pass. They could also be used to estimate partition functions and free energies, conditional probabilities of supersets (supergraphs) given a subset (subgraph), as well as marginal distributions over all supersets (supergraphs) of a given set (graph). We introduce variations enabling the estimation of entropy and mutual information, sampling from a Pareto frontier, connections to reward-maximizing policies, and extensions to stochastic environments, continuous actions and modular energy functions.
2021-11-17
ArXiv (preprint)
arxiv.org
arxiv.org
GFlowNet Foundations
Yoshua Bengio
Tristan Deleu
Edward J Hu
Salem Lahlou
Mo Tiwari
Emmanuel Bengio
2021-11-17
ArXiv (preprint)
arxiv.org
arxiv.org
GFlowNet Foundations
Yoshua Bengio
Tristan Deleu
Edward J Hu
Salem Lahlou
Mo Tiwari
Emmanuel Bengio
2021-11-17
ArXiv (preprint)
arxiv.org
arxiv.org
GFlowNet Foundations
Yoshua Bengio
Tristan Deleu
Edward J Hu
Salem Lahlou
Mo Tiwari
Emmanuel Bengio
Generative Flow Networks (GFlowNets) have been introduced as a method to sample a diverse set of candidates in an active learning context, w… (see more)ith a training objective that makes them approximately sample in proportion to a given reward function. In this paper, we show a number of additional theoretical properties of GFlowNets. They can be used to estimate joint probability distributions and the corresponding marginal distributions where some variables are unspecified and, of particular interest, can represent distributions over composite objects like sets and graphs. GFlowNets amortize the work typically done by computationally expensive MCMC methods in a single but trained generative pass. They could also be used to estimate partition functions and free energies, conditional probabilities of supersets (supergraphs) given a subset (subgraph), as well as marginal distributions over all supersets (supergraphs) of a given set (graph). We introduce variations enabling the estimation of entropy and mutual information, sampling from a Pareto frontier, connections to reward-maximizing policies, and extensions to stochastic environments, continuous actions and modular energy functions.
2021-11-17
ArXiv (preprint)
arxiv.org
arxiv.org
GFlowNet Foundations
Yoshua Bengio
Tristan Deleu
Edward J Hu
Salem Lahlou
Mo Tiwari
Emmanuel Bengio
2021-11-17
ArXiv (preprint)
arxiv.org
arxiv.org
GFlowNet Foundations
Yoshua Bengio
Tristan Deleu
Edward J Hu
Salem Lahlou
Mo Tiwari
Emmanuel Bengio
Generative Flow Networks (GFlowNets) have been introduced as a method to sample a diverse set of candidates in an active learning context, w… (see more)ith a training objective that makes them approximately sample in proportion to a given reward function. In this paper, we show a number of additional theoretical properties of GFlowNets. They can be used to estimate joint probability distributions and the corresponding marginal distributions where some variables are unspecified and, of particular interest, can represent distributions over composite objects like sets and graphs. GFlowNets amortize the work typically done by computationally expensive MCMC methods in a single but trained generative pass. They could also be used to estimate partition functions and free energies, conditional probabilities of supersets (supergraphs) given a subset (subgraph), as well as marginal distributions over all supersets (supergraphs) of a given set (graph). We introduce variations enabling the estimation of entropy and mutual information, sampling from a Pareto frontier, connections to reward-maximizing policies, and extensions to stochastic environments, continuous actions and modular energy functions.
2021-11-17
ArXiv (preprint)
arxiv.org
arxiv.org
GFlowNet Foundations
Yoshua Bengio
Tristan Deleu
Edward J Hu
Salem Lahlou
Mo Tiwari
Emmanuel Bengio
2021-11-17
ArXiv (preprint)
arxiv.org
arxiv.org
GFlowNet Foundations
Yoshua Bengio
Tristan Deleu
Edward J Hu
Salem Lahlou
Mo Tiwari
Emmanuel Bengio
2021-11-17
ArXiv (preprint)
arxiv.org
arxiv.org
Structured Sparsity Inducing Adaptive Optimizers for Deep Learning
Tristan Deleu
Yoshua Bengio
The parameters of a neural network are naturally organized in groups, some of which might not contribute to its overall performance. To prun… (see more)e out unimportant groups of parameters, we can include some non-differentiable penalty to the objective function, and minimize it using proximal gradient methods. In this paper, we derive the weighted proximal operator, which is a necessary component of these proximal methods, of two structured sparsity inducing penalties. Moreover, they can be approximated efficiently with a numerical solver, and despite this approximation, we prove that existing convergence guarantees are preserved when these operators are integrated as part of a generic adaptive proximal method. Finally, we show that this adaptive method, together with the weighted proximal operators derived here, is indeed capable of finding solutions with structure in their sparsity patterns, on representative examples from computer vision and natural language processing.
2021-02-07
ArXiv (preprint)
arxiv.org
arxiv.org