A Unifying Tensor View for Lightweight CNNs
Abstract
Despite the decomposition of convolutional kernels for lightweight CNNs being well studied, existing works that rely on tensor network diagrams or hyperdimensional abstraction lack geometry intuition. This work devises a new perspective by linking a 3D-reshaped kernel tensor to its various slice-wise and rank-1 decompositions, permitting a straightforward connection between various tensor approximations and efficient CNN modules. Specifically, it is discovered that a pointwise-depthwise-pointwise (PDP) configuration constitutes a viable construct for lightweight CNNs. Moreover, a novel link to the latest ShiftNet is established, inspiring a first-ever shift layer pruning that achieves nearly 50% compression with < 1% drop in accuracy for ShiftResNet.
Cite
@article{arxiv.2312.09922,
title = {A Unifying Tensor View for Lightweight CNNs},
author = {Jason Chun Lok Li and Rui Lin and Jiajun Zhou and Edmund Yin Mun Lam and Ngai Wong},
journal= {arXiv preprint arXiv:2312.09922},
year = {2023}
}
Comments
4 pages, 3 figures, accepted in 2023 IEEE 15th International Conference on ASIC (ASICON 2023)