Bayesian Linear Regression on Deep Representations
Abstract
A simple approach to obtaining uncertainty-aware neural networks for regression is to do Bayesian linear regression (BLR) on the representation from the last hidden layer. Recent work [Riquelme et al., 2018, Azizzadenesheli et al., 2018] indicates that the method is promising, though it has been limited to homoscedastic noise. In this paper, we propose a novel variation that enables the method to flexibly model heteroscedastic noise. The method is benchmarked against two prominent alternative methods on a set of standard datasets, and finally evaluated as an uncertainty-aware model in model-based reinforcement learning. Our experiments indicate that the method is competitive with standard ensembling, and ensembles of BLR outperforms the methods we compared to.
Cite
@article{arxiv.1912.06760,
title = {Bayesian Linear Regression on Deep Representations},
author = {John Moberg and Lennart Svensson and Juliano Pinto and Henk Wymeersch},
journal= {arXiv preprint arXiv:1912.06760},
year = {2019}
}
Comments
4th workshop on Bayesian Deep Learning (NeurIPS 2019), Vancouver, Canada