Parameter pruning is a promising approach for CNN compression and acceleration by eliminating redundant model parameters with tolerable performance loss. Despite its effectiveness, existing regularization-based parameter pruning methods usually drive weights towards zero with large and constant regularization factors, which neglects the fact that the expressiveness of CNNs is fragile and needs a more gentle way of regularization for the networks to adapt during pruning. To solve this problem, we propose a new regularization-based pruning method (named IncReg) to incrementally assign different regularization factors to different weight groups based on their relative importance, whose effectiveness is proved on popular CNNs compared with state-of-the-art methods.
@article{arxiv.1811.08390,
title = {Structured Pruning for Efficient ConvNets via Incremental Regularization},
author = {Huan Wang and Qiming Zhang and Yuehai Wang and Haoji Hu},
journal= {arXiv preprint arXiv:1811.08390},
year = {2018}
}
Comments
Accepted by NIPS 2018 workshop on "Compact Deep Neural Network Representation with Industrial Applications"