Variational f-divergence Minimization
Abstract
Probabilistic models are often trained by maximum likelihood, which corresponds to minimizing a specific f-divergence between the model and data distribution. In light of recent successes in training Generative Adversarial Networks, alternative non-likelihood training criteria have been proposed. Whilst not necessarily statistically efficient, these alternatives may better match user requirements such as sharp image generation. A general variational method for training probabilistic latent variable models using maximum likelihood is well established; however, how to train latent variable models using other f-divergences is comparatively unknown. We discuss a variational approach that, when combined with the recently introduced Spread Divergence, can be applied to train a large class of latent variable models using any f-divergence.
Cite
@article{arxiv.1907.11891,
title = {Variational f-divergence Minimization},
author = {Mingtian Zhang and Thomas Bird and Raza Habib and Tianlin Xu and David Barber},
journal= {arXiv preprint arXiv:1907.11891},
year = {2024}
}