Optimal Brain Connection: Towards Efficient Structural Pruning
Abstract
Structural pruning has been widely studied for its effectiveness in compressing neural networks. However, existing methods often neglect the interconnections among parameters. To address this limitation, this paper proposes a structural pruning framework termed Optimal Brain Connection. First, we introduce the Jacobian Criterion, a first-order metric for evaluating the saliency of structural parameters. Unlike existing first-order methods that assess parameters in isolation, our criterion explicitly captures both intra-component interactions and inter-layer dependencies. Second, we propose the Equivalent Pruning mechanism, which utilizes autoencoders to retain the contributions of all original connection--including pruned ones--during fine-tuning. Experimental results demonstrate that the Jacobian Criterion outperforms several popular metrics in preserving model performance, while the Equivalent Pruning mechanism effectively mitigates performance degradation after fine-tuning. Code: https://github.com/ShaowuChen/Optimal_Brain_Connection
Cite
@article{arxiv.2508.05521,
title = {Optimal Brain Connection: Towards Efficient Structural Pruning},
author = {Shaowu Chen and Wei Ma and Binhua Huang and Qingyuan Wang and Guoxin Wang and Weize Sun and Lei Huang and Deepu John},
journal= {arXiv preprint arXiv:2508.05521},
year = {2025}
}