Embedding Differentiable Sparsity into Deep Neural Network

Yongjin Lee

Embedding Differentiable Sparsity into Deep Neural Network

Machine Learning 2020-06-25 v1 Machine Learning

Authors: Yongjin Lee

Abstract

In this paper, we propose embedding sparsity into the structure of deep neural networks, where model parameters can be exactly zero during training with the stochastic gradient descent. Thus, it can learn the sparsified structure and the weights of networks simultaneously. The proposed approach can learn structured as well as unstructured sparsity.

Keywords

sparse learning deep neural networks neural network training

Cite

@article{arxiv.2006.13716,
  title  = {Embedding Differentiable Sparsity into Deep Neural Network},
  author = {Yongjin Lee},
  journal= {arXiv preprint arXiv:2006.13716},
  year   = {2020}
}

Comments

arXiv admin note: text overlap with arXiv:1910.03201

Related papers

View all related →