VIB is Half Bayes
Machine Learning
2020-11-18 v1 Machine Learning
Abstract
In discriminative settings such as regression and classification there are two random variables at play, the inputs X and the targets Y. Here, we demonstrate that the Variational Information Bottleneck can be viewed as a compromise between fully empirical and fully Bayesian objectives, attempting to minimize the risks due to finite sampling of Y only. We argue that this approach provides some of the benefits of Bayes while requiring only some of the work.
Cite
@article{arxiv.2011.08711,
title = {VIB is Half Bayes},
author = {Alexander A Alemi and Warren R Morningstar and Ben Poole and Ian Fischer and Joshua V Dillon},
journal= {arXiv preprint arXiv:2011.08711},
year = {2020}
}