Portrait of Nikolay Malkin

Nikolay Malkin

Collaborating Alumni - Université de Montréal
Supervisor
Yoshua Bengio
Research Topics
Computer Vision
Deep Learning
Generative Models
Natural Language Processing
Probabilistic Models
Reasoning
Reinforcement Learning
Website
Google Scholar
GitHub

Publications

Unifying Generative Models with GFlowNets
Dinghuai Zhang
Ricky T. Q. Chen
Nikolay Malkin
Yoshua Bengio
There are many frameworks for deep generative modeling, each often presented with their own specific training algorithms and inference metho… (see more)ds. Here, we demonstrate the connections between existing deep generative models and the recently introduced GFlowNet framework, a probabilistic inference machine which treats sampling as a decision-making process. This analysis sheds light on their overlapping traits and provides a unifying viewpoint through the lens of learning with Markovian trajectories. Our framework provides a means for unifying training and inference algorithms, and provides a route to shine a unifying light over many generative models. Beyond this, we provide a practical and experimentally verified recipe for improving generative modeling with insights from the GFlowNet perspective.
2022-09-06
ArXiv (preprint)
doi.org
arxiv.org
Generative Flow Networks for Discrete Probabilistic Modeling
Dinghuai Zhang
Nikolay Malkin
Zhen Liu
Alexandra Volokhova
Aaron Courville
Yoshua Bengio
We present energy-based generative flow networks (EB-GFN), a novel probabilistic modeling algorithm for high-dimensional discrete data. Buil… (see more)ding upon the theory of generative flow networks (GFlowNets), we model the generation process by a stochastic data construction policy and thus amortize expensive MCMC exploration into a fixed number of actions sampled from a GFlowNet. We show how GFlowNets can approximately perform large-block Gibbs sampling to mix between modes. We propose a framework to jointly train a GFlowNet with an energy function, so that the GFlowNet learns to sample from the energy distribution, while the energy learns with an approximate MLE objective with negative samples from the GFlowNet. We demonstrate EB-GFN's effectiveness on various probabilistic modeling tasks. Code is publicly available at https://github.com/zdhNarsil/EB_GFN.
2022-06-28
Proceedings of the 39th International Conference on Machine Learning (published)
proceedings.mlr.press
arxiv.org
Trajectory Balance: Improved Credit Assignment in GFlowNets
Nikolay Malkin
Moksh J. Jain
Emmanuel Bengio
Chen Sun
Yoshua Bengio
Generative flow networks (GFlowNets) are a method for learning a stochastic policy for generating compositional objects, such as graphs or s… (see more)trings, from a given unnormalized density by sequences of actions, where many possible action sequences may lead to the same object. We find previously proposed learning objectives for GFlowNets, flow matching and detailed balance, which are analogous to temporal difference learning, to be prone to inefficient credit propagation across long action sequences. We thus propose a new learning objective for GFlowNets, trajectory balance, as a more efficient alternative to previously used objectives. We prove that any global minimizer of the trajectory balance objective can define a policy that samples exactly from the target distribution. In experiments on four distinct domains, we empirically demonstrate the benefits of the trajectory balance objective for GFlowNet convergence, diversity of generated samples, and robustness to long action sequences and large action spaces.
openreview.net