Generalization Gap in Amortized Inference
Machine Learning
2022-10-18 v2 Machine Learning
Abstract
The ability of likelihood-based probabilistic models to generalize to unseen data is central to many machine learning applications such as lossless compression. In this work, we study the generalization of a popular class of probabilistic model - the Variational Auto-Encoder (VAE). We discuss the two generalization gaps that affect VAEs and show that overfitting is usually dominated by amortized inference. Based on this observation, we propose a new training objective that improves the generalization of amortized inference. We demonstrate how our method can improve performance in the context of image modeling and lossless compression.
Cite
@article{arxiv.2205.11640,
title = {Generalization Gap in Amortized Inference},
author = {Mingtian Zhang and Peter Hayes and David Barber},
journal= {arXiv preprint arXiv:2205.11640},
year = {2022}
}