Gaussian Gated Linear Networks
Abstract
We propose the Gaussian Gated Linear Network (G-GLN), an extension to the recently proposed GLN family of deep neural networks. Instead of using backpropagation to learn features, GLNs have a distributed and local credit assignment mechanism based on optimizing a convex objective. This gives rise to many desirable properties including universality, data-efficient online learning, trivial interpretability and robustness to catastrophic forgetting. We extend the GLN framework from classification to multiple regression and density modelling by generalizing geometric mixing to a product of Gaussian densities. The G-GLN achieves competitive or state-of-the-art performance on several univariate and multivariate regression benchmarks, and we demonstrate its applicability to practical tasks including online contextual bandits and density estimation via denoising.
Cite
@article{arxiv.2006.05964,
title = {Gaussian Gated Linear Networks},
author = {David Budden and Adam Marblestone and Eren Sezener and Tor Lattimore and Greg Wayne and Joel Veness},
journal= {arXiv preprint arXiv:2006.05964},
year = {2020}
}