基于条件互信息的深度卷积神经网络剪枝
摘要
卷积神经网络（CNNs）在图像分类任务中取得高性能，但由于模型大小较大，在资源受限的硬件上部署具有挑战性。为了解决这个问题，我们利用互信息（Mutual Information），这是一种度量方法，提供了有关深度学习模型如何保留和处理信息的有价值见解，通过衡量输入特征或输出标签与网络各层之间的共享信息。In this study, we propose a structured filter-pruning approach for CNNs that identifies and selectively retains the most informative features in each layer. Our approach successively evaluates each layer by ranking the importance of its feature maps based on Conditional Mutual Information (CMI) values, computed using a matrix-based Renyi {\alpha}-order entropy numerical method. We propose several formulations of CMI to capture correlation among features across different layers. We then develop various strategies to determine the cutoff point for CMI values to prune unimportant features. This approach allows parallel pruning in both forward and backward directions and significantly reduces model size while preserving accuracy. Tested on the VGG16 architecture with the CIFAR-10 dataset, the proposed method reduces the number of filters by more than a third, with only a 0.32% drop in test accuracy.
引用
@article{arxiv.2411.18578,
title = {Pruning Deep Convolutional Neural Network Using Conditional Mutual Information},
author = {Tien Vu-Van and Dat Du Thanh and Nguyen Ho and Mai Vu},
journal= {arXiv preprint arXiv:2411.18578},
year = {2024}
}