This paper presents a practical application of Relative Trajectory Balance (RTB), a recently introduced off-policy reinforcement learning (RL) objective that can asymptotically solve Bayesian inverse problems optimally. We extend the original work by using RTB to train conditional diffusion model posteriors from pretrained unconditional priors for challenging linear and non-linear inverse problems in vision, and science. We use the objective alongside techniques such as off-policy backtracking exploration to improve training. Importantly, our results show that existing training-free diffusion posterior methods struggle to perform effective posterior inference in latent space due to inherent biases.
@article{arxiv.2503.09746,
title = {Solving Bayesian inverse problems with diffusion priors and off-policy RL},
author = {Luca Scimeca and Siddarth Venkatraman and Moksh Jain and Minsu Kim and Marcin Sendera and Mohsin Hasan and Luke Rowe and Sarthak Mittal and Pablo Lemos and Emmanuel Bengio and Alexandre Adam and Jarrid Rector-Brooks and Yashar Hezaveh and Laurence Perreault-Levasseur and Yoshua Bengio and Glen Berseth and Nikolay Malkin},
journal= {arXiv preprint arXiv:2503.09746},
year = {2025}
}
Comments
Accepted as workshop paper at DeLTa workshop, ICLR 2025. arXiv admin note: substantial text overlap with arXiv:2405.20971