The energy consumption of large-scale ML models is dominated by data movement, shuffling billions of parameters across memory hierarchies and data centers. Sparsification offers a principled way to mitigate these costs by pruning redundant weights and activations, thereby reducing data movement. Effective sparsification to prune redundant parameters is still challenging: existing methods incur significant accuracy degradation, performance overhead, or both. We introduce (Bl)ock (a)nd (S)parse (T)ransformers (BLaST), a general, robust, and reliable method for sparsification, applicable to linear layers in all settings. Our method iteratively sparsifies weight matrices into a block sparsity pattern suitable for efficient sparse matrix-matrix (SpMM) multiplication. BLaST achieves up to 95% sparsity in MLP weights with negligible accuracy loss (majority <2.25%). We show a 2.2x inference speedup for Llama 3.2 with 16 GPUs, and up to 4.45x reduction in inference memory footprint resulting in a 2.9x reduction in GPU setup and operating costs.
@article{arxiv.2507.03117,
title = {BLaST: High Performance Inference and Pretraining using BLock Sparse Transformers},
author = {Patrik Okanovic and Sameer Deshmukh and Grzegorz Kwasniewski and Yi Zhu and Haruto Fujii and Sakina Fatima and Maciej Besta and Kentaro Katayama and Takumi Honda and Yusuke Nagasaka and Torsten Hoefler},
journal= {arXiv preprint arXiv:2507.03117},
year = {2025}
}