A novel method to propagate uncertainty through the soft-thresholding nonlinearity is proposed in this paper. At every layer the current distribution of the target vector is represented as a spike and slab distribution, which represents the probabilities of each variable being zero, or Gaussian-distributed. Using the proposed method of uncertainty propagation, the gradients of the logarithms of normalisation constants are derived, that can be used to update a weight distribution. A novel Bayesian neural network for sparse coding is designed utilising both the proposed method of uncertainty propagation and Bayesian inference algorithm.
@article{arxiv.1811.12465,
title = {Uncertainty propagation in neural networks for sparse coding},
author = {Danil Kuzin and Olga Isupova and Lyudmila Mihaylova},
journal= {arXiv preprint arXiv:1811.12465},
year = {2018}
}
Comments
Presented at the third workshop on Bayesian Deep Learning (NeurIPS 2018)