Competitive Training of Mixtures of Independent Deep Generative Models
Abstract
A common assumption in causal modeling posits that the data is generated by a set of independent mechanisms, and algorithms should aim to recover this structure. Standard unsupervised learning, however, is often concerned with training a single model to capture the overall distribution or aspects thereof. Inspired by clustering approaches, we consider mixtures of implicit generative models that ``disentangle'' the independent generative mechanisms underlying the data. Relying on an additional set of discriminators, we propose a competitive training procedure in which the models only need to capture the portion of the data distribution from which they can produce realistic samples. As a by-product, each model is simpler and faster to train. We empirically show that our approach splits the training distribution in a sensible way and increases the quality of the generated samples.
Keywords
Cite
@article{arxiv.1804.11130,
title = {Competitive Training of Mixtures of Independent Deep Generative Models},
author = {Francesco Locatello and Damien Vincent and Ilya Tolstikhin and Gunnar Rätsch and Sylvain Gelly and Bernhard Schölkopf},
journal= {arXiv preprint arXiv:1804.11130},
year = {2019}
}