Deep Probabilistic Ensembles: Approximate Variational Inference through KL Regularization
Machine Learning
2018-12-04 v2 Machine Learning
Abstract
In this paper, we introduce Deep Probabilistic Ensembles (DPEs), a scalable technique that uses a regularized ensemble to approximate a deep Bayesian Neural Network (BNN). We do so by incorporating a KL divergence penalty term into the training objective of an ensemble, derived from the evidence lower bound used in variational inference. We evaluate the uncertainty estimates obtained from our models for active learning on visual classification. Our approach steadily improves upon active learning baselines as the annotation budget is increased.
Cite
@article{arxiv.1811.02640,
title = {Deep Probabilistic Ensembles: Approximate Variational Inference through KL Regularization},
author = {Kashyap Chitta and Jose M. Alvarez and Adam Lesnikowski},
journal= {arXiv preprint arXiv:1811.02640},
year = {2018}
}
Comments
Workshop on Bayesian Deep Learning (NeurIPS 2018)