Portrait of Dinghuai Zhang

Dinghuai Zhang

PhD - Université de Montréal
Supervisor
Aaron Courville
Co-supervisor
Yoshua Bengio
Research Topics
Generative Models
Probabilistic Models
Website

Publications

Local Search GFlowNets
Minsu Kim
Taeyoung YUN
Emmanuel Bengio
Dinghuai Zhang
Yoshua Bengio
Sungsoo Ahn
Jinkyoo Park
2023-10-04
ArXiv (preprint)
doi.org
arxiv.org
Local Search GFlowNets
Minsu Kim
Taeyoung YUN
Emmanuel Bengio
Dinghuai Zhang
Yoshua Bengio
Sungsoo Ahn
Jinkyoo Park
Generative Flow Networks (GFlowNets) are amortized sampling methods that learn a distribution over discrete objects proportional to their re… (see more)wards. GFlowNets exhibit a remarkable ability to generate diverse samples, yet occasionally struggle to consistently produce samples with high rewards due to over-exploration on wide sample space. This paper proposes to train GFlowNets with local search, which focuses on exploiting high-rewarded sample space to resolve this issue. Our main idea is to explore the local neighborhood via backtracking and reconstruction guided by backward and forward policies, respectively. This allows biasing the samples toward high-reward solutions, which is not possible for a typical GFlowNet solution generation scheme, which uses the forward policy to generate the solution from scratch. Extensive experiments demonstrate a remarkable performance improvement in several biochemical tasks. Source code is available: https://github.com/dbsxodud-11/ls_gfn.
2023-10-04
ArXiv (preprint)
doi.org
arxiv.org
Local Search GFlowNets
Minsu Kim
Taeyoung YUN
Emmanuel Bengio
Dinghuai Zhang
Yoshua Bengio
Sungsoo Ahn
Jinkyoo Park
2023-10-04
ArXiv (preprint)
doi.org
arxiv.org
Local Search GFlowNets
Minsu Kim
Taeyoung YUN
Emmanuel Bengio
Dinghuai Zhang
Yoshua Bengio
Sungsoo Ahn
Jinkyoo Park
2023-10-04
ArXiv (preprint)
doi.org
arxiv.org
Local Search GFlowNets
Minsu Kim
Taeyoung YUN
Emmanuel Bengio
Dinghuai Zhang
Yoshua Bengio
Sungsoo Ahn
Jinkyoo Park
Generative Flow Networks (GFlowNets) are amortized sampling methods that learn a distribution over discrete objects proportional to their re… (see more)wards. GFlowNets exhibit a remarkable ability to generate diverse samples, yet occasionally struggle to consistently produce samples with high rewards due to over-exploration on wide sample space. This paper proposes to train GFlowNets with local search, which focuses on exploiting high-rewarded sample space to resolve this issue. Our main idea is to explore the local neighborhood via backtracking and reconstruction guided by backward and forward policies, respectively. This allows biasing the samples toward high-reward solutions, which is not possible for a typical GFlowNet solution generation scheme, which uses the forward policy to generate the solution from scratch. Extensive experiments demonstrate a remarkable performance improvement in several biochemical tasks. Source code is available: https://github.com/dbsxodud-11/ls_gfn.
2023-10-04
ArXiv (preprint)
doi.org
arxiv.org
Local Search GFlowNets
Minsu Kim
Taeyoung YUN
Emmanuel Bengio
Dinghuai Zhang
Yoshua Bengio
Sungsoo Ahn
Jinkyoo Park
2023-10-04
ArXiv (preprint)
doi.org
arxiv.org
Local Search GFlowNets
Minsu Kim
Taeyoung YUN
Emmanuel Bengio
Dinghuai Zhang
Yoshua Bengio
Sungsoo Ahn
Jinkyoo Park
Generative Flow Networks (GFlowNets) are amortized sampling methods that learn a distribution over discrete objects proportional to their re… (see more)wards. GFlowNets exhibit a remarkable ability to generate diverse samples, yet occasionally struggle to consistently produce samples with high rewards due to over-exploration on wide sample space. This paper proposes to train GFlowNets with local search, which focuses on exploiting high-rewarded sample space to resolve this issue. Our main idea is to explore the local neighborhood via backtracking and reconstruction guided by backward and forward policies, respectively. This allows biasing the samples toward high-reward solutions, which is not possible for a typical GFlowNet solution generation scheme, which uses the forward policy to generate the solution from scratch. Extensive experiments demonstrate a remarkable performance improvement in several biochemical tasks. Source code is available: https://github.com/dbsxodud-11/ls_gfn.
2023-10-04
ArXiv (preprint)
doi.org
arxiv.org
Local Search GFlowNets
Minsu Kim
Taeyoung YUN
Emmanuel Bengio
Dinghuai Zhang
Yoshua Bengio
Sungsoo Ahn
Jinkyoo Park
Generative Flow Networks (GFlowNets) are amortized sampling methods that learn a distribution over discrete objects proportional to their re… (see more)wards. GFlowNets exhibit a remarkable ability to generate diverse samples, yet occasionally struggle to consistently produce samples with high rewards due to over-exploration on wide sample space. This paper proposes to train GFlowNets with local search, which focuses on exploiting high-rewarded sample space to resolve this issue. Our main idea is to explore the local neighborhood via backtracking and reconstruction guided by backward and forward policies, respectively. This allows biasing the samples toward high-reward solutions, which is not possible for a typical GFlowNet solution generation scheme, which uses the forward policy to generate the solution from scratch. Extensive experiments demonstrate a remarkable performance improvement in several biochemical tasks. Source code is available: https://github.com/dbsxodud-11/ls_gfn.
2023-10-04
ArXiv (preprint)
doi.org
arxiv.org
Local Search GFlowNets
Minsu Kim
Taeyoung YUN
Emmanuel Bengio
Dinghuai Zhang
Yoshua Bengio
Sungsoo Ahn
Jinkyoo Park
Generative Flow Networks (GFlowNets) are amortized sampling methods that learn a distribution over discrete objects proportional to their re… (see more)wards. GFlowNets exhibit a remarkable ability to generate diverse samples, yet occasionally struggle to consistently produce samples with high rewards due to over-exploration on wide sample space. This paper proposes to train GFlowNets with local search, which focuses on exploiting high-rewarded sample space to resolve this issue. Our main idea is to explore the local neighborhood via backtracking and reconstruction guided by backward and forward policies, respectively. This allows biasing the samples toward high-reward solutions, which is not possible for a typical GFlowNet solution generation scheme, which uses the forward policy to generate the solution from scratch. Extensive experiments demonstrate a remarkable performance improvement in several biochemical tasks. Source code is available: https://github.com/dbsxodud-11/ls_gfn.
2023-10-04
ArXiv (preprint)
doi.org
arxiv.org
Let the Flows Tell: Solving Graph Combinatorial Problems with GFlowNets
Dinghuai Zhang
Hanjun Dai
Nikolay Malkin
Aaron Courville
Yoshua Bengio
Ling Pan
openreview.net
Better Training of GFlowNets with Local Credit and Incomplete Trajectories
Ling Pan
Nikolay Malkin
Dinghuai Zhang
Yoshua Bengio
Generative Flow Networks or GFlowNets are related to Monte-Carlo Markov chain methods (as they sample from a distribution specified by an en… (see more)ergy function), reinforcement learning (as they learn a policy to sample composed objects through a sequence of steps), generative models (as they learn to represent and sample from a distribution) and amortized variational methods (as they can be used to learn to approximate and sample from an otherwise intractable posterior, given a prior and a likelihood). They are trained to generate an object
2023-07-03
Proceedings of the 40th International Conference on Machine Learning (published)
doi.org
openreview.net
Let the Flows Tell: Solving Graph Combinatorial Optimization Problems with GFlowNets
Dinghuai Zhang
Hanjun Dai
Nikolay Malkin
Aaron Courville
Yoshua Bengio
Ling Pan
Combinatorial optimization (CO) problems are often NP-hard and thus out of reach for exact algorithms, making them a tempting domain to appl… (see more)y machine learning methods. The highly structured constraints in these problems can hinder either optimization or sampling directly in the solution space. On the other hand, GFlowNets have recently emerged as a powerful machinery to efficiently sample from composite unnormalized densities sequentially and have the potential to amortize such solution-searching processes in CO, as well as generate diverse solution candidates. In this paper, we design Markov decision processes (MDPs) for different combinatorial problems and propose to train conditional GFlowNets to sample from the solution space. Efficient training techniques are also developed to benefit long-range credit assignment. Through extensive experiments on a variety of different CO tasks with synthetic and realistic data, we demonstrate that GFlowNet policies can efficiently find high-quality solutions. Our implementation is open-sourced at https://github.com/zdhNarsil/GFlowNet-CombOpt.
2023-05-26
ArXiv (preprint)
doi.org
arxiv.org