Conditionally Gaussian PAC-Bayes
Machine Learning
2022-07-01 v2 Machine Learning
Abstract
Recent studies have empirically investigated different methods to train stochastic neural networks on a classification task by optimising a PAC-Bayesian bound via stochastic gradient descent. Most of these procedures need to replace the misclassification error with a surrogate loss, leading to a mismatch between the optimisation objective and the actual generalisation bound. The present paper proposes a novel training algorithm that optimises the PAC-Bayesian bound, without relying on any surrogate loss. Empirical results show that this approach outperforms currently available PAC-Bayesian training methods.
Cite
@article{arxiv.2110.11886,
title = {Conditionally Gaussian PAC-Bayes},
author = {Eugenio Clerico and George Deligiannidis and Arnaud Doucet},
journal= {arXiv preprint arXiv:2110.11886},
year = {2022}
}