On the Mathematics of Diffusion Models
Machine Learning
2023-03-07 v3 Artificial Intelligence
Probability
Abstract
This paper gives direct derivations of the differential equations and likelihood formulas of diffusion models assuming only knowledge of Gaussian distributions. A VAE analysis derives both forward and backward stochastic differential equations (SDEs) as well as non-variational integral expressions for likelihood formulas. A score-matching analysis derives the reverse diffusion ordinary differential equation (ODE) and a family of reverse-diffusion SDEs parameterized by noise level. The paper presents the mathematics directly with attributions saved for a final section.
Keywords
Cite
@article{arxiv.2301.11108,
title = {On the Mathematics of Diffusion Models},
author = {David McAllester},
journal= {arXiv preprint arXiv:2301.11108},
year = {2023}
}