Siddarth Venkatraman

openreview.net

Solving Bayesian inverse problems with diffusion priors and off-policy RL

Luca Scimeca

Laurence Perreault-Levasseur

Moksh J. Jain

Minsu Kim

Yoshua Bengio

Glen Berseth

Nikolay Malkin

This paper presents a practical application of Relative Trajectory Balance (RTB), a recently introduced off-policy reinforcement learning (R… (voir plus)L) objective that can asymptotically solve Bayesian inverse problems optimally. We extend the original work by using RTB to train conditional diffusion model posteriors from pretrained unconditional priors for challenging linear and non-linear inverse problems in vision, and science. We use the objective alongside techniques such as off-policy backtracking exploration to improve training. Importantly, our results show that existing training-free diffusion posterior methods struggle to perform effective posterior inference in latent space due to inherent biases.

2025-03-06

ICLR.cc/2025/Workshop/DeLTa (poster)

openreview.net

Outsourced diffusion sampling: Efficient posterior inference in latent spaces of generative models

Minsu Kim

Any well-behaved generative model over a variable …

2025-02-10

ArXiv (prépublication)

Trajectory Balance with Asynchrony: Decoupling Exploration and Learning for Fast, Scalable LLM Post-Training

Brian R. Bartoldson

James Diffenderfer

Moksh J. Jain

Tal Ben-Nun

Seanie Lee

Minsu Kim

Johan Samir Obando Ceron

Yoshua Bengio

Bhavya Kailkhura

2025-01-01

arXiv.org (prépublication)

Trajectory Balance with Asynchrony: Decoupling Exploration and Learning for Fast, Scalable LLM Post-Training

Brian R. Bartoldson

James Diffenderfer

Moksh J. Jain

Tal Ben-Nun

Seanie Lee

Minsu Kim

Johan Samir Obando Ceron

Yoshua Bengio

Bhavya Kailkhura

2025-01-01

arXiv.org (prépublication)

Amortizing intractable inference in diffusion models for vision, language, and control

Moksh J. Jain

Minsu Kim

Diffusion models have emerged as effective distribution estimators in vision, language, and reinforcement learning, but their use as priors … (voir plus)in downstream tasks poses an intractable posterior inference problem. This paper studies amortized sampling of the posterior over data,

2024-09-25

NeurIPS.cc/2024/Conference (poster)

openreview.net

Amortizing intractable inference in diffusion models for vision, language, and control

Moksh J. Jain

Minsu Kim

Diffusion models have emerged as effective distribution estimators in vision, language, and reinforcement learning, but their use as priors … (voir plus)in downstream tasks poses an intractable posterior inference problem. This paper studies amortized sampling of the posterior over data,

2024-05-31

ArXiv (prépublication)

Amortizing intractable inference in diffusion models for vision, language, and control

Moksh J. Jain

Minsu Kim

Diffusion models have emerged as effective distribution estimators in vision, language, and reinforcement learning, but their use as priors … (voir plus)in downstream tasks poses an intractable posterior inference problem. This paper studies amortized sampling of the posterior over data,

2024-05-31

ArXiv (prépublication)

Amortizing intractable inference in diffusion models for vision, language, and control

Moksh J. Jain

Minsu Kim

Diffusion models have emerged as effective distribution estimators in vision, language, and reinforcement learning, but their use as priors … (voir plus)in downstream tasks poses an intractable posterior inference problem. This paper studies amortized sampling of the posterior over data,

2024-05-31

ArXiv (prépublication)

Reasoning with Latent Diffusion in Offline Reinforcement Learning