Generative Adversarial Networks (Goodfellow et al., 2014), a major breakthrough in the field of generative modeling, learn a discriminator to estimate some distance between the target and the candidate distributions. This paper examines mathematical issues regarding the way the gradients for the generative model are computed in this context, and notably how to take into account how the discriminator itself depends on the generator parameters. A unifying methodology is presented to define mathematically sound training objectives for generative models taking this dependency into account in a robust way, covering both GAN, VAE and some GAN variants as particular cases.
@article{arxiv.1807.01290,
title = {New Losses for Generative Adversarial Learning},
author = {Victor Berger and Michèle Sebag},
journal= {arXiv preprint arXiv:1807.01290},
year = {2018}
}
Comments
The central result of the paper was based on a wrong assumption: the term in the loss capturing the variation of the optimal discriminator with relation to the generator can be proved to be always zero using the Envelope Theorem