Quantization is spearheading the increase in performance and efficiency of neural network computing systems making headway into commodity hardware. We present SWIS - Shared Weight bIt Sparsity, a quantization framework for efficient neural network inference acceleration delivering improved performance and storage compression through an offline weight decomposition and scheduling algorithm. SWIS can achieve up to 54.3% (19.8%) point accuracy improvement compared to weight truncation when quantizing MobileNet-v2 to 4 (2) bits post-training (with retraining) showing the strength of leveraging shared bit-sparsity in weights. SWIS accelerator gives up to 6x speedup and 1.9x energy improvement overstate of the art bit-serial architectures.
@article{arxiv.2103.01308,
title = {SWIS -- Shared Weight bIt Sparsity for Efficient Neural Network Acceleration},
author = {Shurui Li and Wojciech Romaszkan and Alexander Graening and Puneet Gupta},
journal= {arXiv preprint arXiv:2103.01308},
year = {2021}
}
Comments
8 pages, 6 figures, accepted as a full-length paper at the 2021 TinyML Research Symposium (https://openreview.net/group?id=tinyml.org/tinyML/2021/Research_Symposium)