Learning in uncertain, noisy, or adversarial environments is a challenging task for deep neural networks (DNNs). We propose a new theoretically grounded and efficient approach for robust learning that builds upon Bayesian estimation and Variational Inference. We formulate the problem of density propagation through layers of a DNN and solve it using an Ensemble Density Propagation (EnDP) scheme. The EnDP approach allows us to propagate moments of the variational probability distribution across the layers of a Bayesian DNN, enabling the estimation of the mean and covariance of the predictive distribution at the output of the model. Our experiments using MNIST and CIFAR-10 datasets show a significant improvement in the robustness of the trained models to random noise and adversarial attacks.
@article{arxiv.2111.05953,
title = {Robust Learning via Ensemble Density Propagation in Deep Neural Networks},
author = {Giuseppina Carannante and Dimah Dera and Ghulam Rasool and Nidhal C. Bouaynaya and Lyudmila Mihaylova},
journal= {arXiv preprint arXiv:2111.05953},
year = {2021}
}
Comments
submitted to 2020 IEEE International Workshop on Machine Learning for Signal Processing