Stochastic Neural Network with Kronecker Flow
Abstract
Recent advances in variational inference enable the modelling of highly structured joint distributions, but are limited in their capacity to scale to the high-dimensional setting of stochastic neural networks. This limitation motivates a need for scalable parameterizations of the noise generation process, in a manner that adequately captures the dependencies among the various parameters. In this work, we address this need and present the Kronecker Flow, a generalization of the Kronecker product to invertible mappings designed for stochastic neural networks. We apply our method to variational Bayesian neural networks on predictive tasks, PAC-Bayes generalization bound estimation, and approximate Thompson sampling in contextual bandits. In all setups, our methods prove to be competitive with existing methods and better than the baselines.
Cite
@article{arxiv.1906.04282,
title = {Stochastic Neural Network with Kronecker Flow},
author = {Chin-Wei Huang and Ahmed Touati and Pascal Vincent and Gintare Karolina Dziugaite and Alexandre Lacoste and Aaron Courville},
journal= {arXiv preprint arXiv:1906.04282},
year = {2020}
}
Comments
Proceedings of the 23rdInternational Conference on ArtificialIntelligence and Statistics (AISTATS) 2020