Current strategies for solving image-based inverse problems apply latent diffusion models to perform posterior sampling.However, almost all approaches make no explicit attempt to explore the solution space, instead drawing only a single sample from a Gaussian distribution from which to generate their solution. In this paper, we introduce a particle-filtering-based framework for a nonlinear exploration of the solution space in the initial stages of reverse SDE methods. Our proposed particle-filtering-based latent diffusion (PFLD) method and proposed problem formulation and framework can be applied to any diffusion-based solution for linear or nonlinear inverse problems. Our experimental results show that PFLD outperforms the SoTA solver PSLD on the FFHQ-1K and ImageNet-1K datasets on inverse problem tasks of super resolution, Gaussian debluring and inpainting.
@article{arxiv.2408.13868,
title = {Particle-Filtering-based Latent Diffusion for Inverse Problems},
author = {Amir Nazemi and Mohammad Hadi Sepanj and Nicholas Pellegrino and Chris Czarnecki and Paul Fieguth},
journal= {arXiv preprint arXiv:2408.13868},
year = {2024}
}
Comments
Mohammad Hadi Sepanj, Nicholas Pellegrino, and Chris Czarnecki contributed equally