Dichotomize and Generalize: PAC-Bayesian Binary Activated Deep Neural Networks
Machine Learning
2020-02-05 v5 Machine Learning
Abstract
We present a comprehensive study of multilayer neural networks with binary activation, relying on the PAC-Bayesian theory. Our contributions are twofold: (i) we develop an end-to-end framework to train a binary activated deep neural network, (ii) we provide nonvacuous PAC-Bayesian generalization bounds for binary activated deep neural networks. Our results are obtained by minimizing the expected loss of an architecture-dependent aggregation of binary activated deep neural networks. Our analysis inherently overcomes the fact that binary activation function is non-differentiable. The performance of our approach is assessed on a thorough numerical experiment protocol on real-life datasets.
Cite
@article{arxiv.1905.10259,
title = {Dichotomize and Generalize: PAC-Bayesian Binary Activated Deep Neural Networks},
author = {Gaël Letarte and Pascal Germain and Benjamin Guedj and François Laviolette},
journal= {arXiv preprint arXiv:1905.10259},
year = {2020}
}