Bayesian Sparsification Methods for Deep Complex-valued Networks
Abstract
With continual miniaturization ever more applications of deep learning can be found in embedded systems, where it is common to encounter data with natural complex domain representation. To this end we extend Sparse Variational Dropout to complex-valued neural networks and verify the proposed Bayesian technique by conducting a large numerical study of the performance-compression trade-off of C-valued networks on two tasks: image recognition on MNIST-like and CIFAR10 datasets and music transcription on MusicNet. We replicate the state-of-the-art result by Trabelsi et al. [2018] on MusicNet with a complex-valued network compressed by 50-100x at a small performance penalty.
Cite
@article{arxiv.2003.11413,
title = {Bayesian Sparsification Methods for Deep Complex-valued Networks},
author = {Ivan Nazarov and Evgeny Burnaev},
journal= {arXiv preprint arXiv:2003.11413},
year = {2020}
}
Comments
Findings and conclusions unchanged. Improved overall presentation and redid the plots with larger markers and annotations. Coherent story about compression, CVNN, BI to SGVB with local reparameterization and additive noise. Better coverage in lit-review, clearer connections of Dropout to Bayes, VD, and pruning