BLASST: Dynamic BLocked Attention Sparsity via Softmax Thresholding
Abstract
The growing demand for long-context inference capabilities in Large Language Models (LLMs) has intensified the computational and memory bottlenecks inherent to the self-attention mechanism. To address this challenge, we introduce BLASST, a drop-in, dynamic sparse attention mechanism that accelerates inference by using only a fixed scalar threshold to skip attention blocks. Our method targets practical inference deployment by removing the barriers to adoption present in existing works. As such, BLASST eliminates training requirements, avoids expensive pre-computation passes, accelerates both prefill and decode across all major attention variants (MHA, GQA, MQA, and MLA), provides optimized support for modern hardware, and easily integrates into existing frameworks. This is achieved by reusing online softmax statistics to identify negligible attention scores, skipping softmax, value block loads, and the subsequent matrix multiplication. We demonstrate the BLASST algorithm by delivering optimized kernels with negligible latency overhead. Our automated threshold calibration procedure reveals a simple inverse relationship between optimal threshold and context length, meaning we require only a single threshold each for prefill and decode per model. Preserving benchmark accuracy, we demonstrate a 1.52x speedup for prefill at 71.9% sparsity and a 1.48x speedup for decode at 73.2% sparsity on modern GPUs.
Cite
@article{arxiv.2512.12087,
title = {BLASST: Dynamic BLocked Attention Sparsity via Softmax Thresholding},
author = {Jiayi Yuan and Cameron Shinn and Kai Xu and Jingze Cui and George Klimiashvili and Guangxuan Xiao and Perkz Zheng and Bo Li and Yuxin Zhou and Zhouhai Ye and Weijie You and Tian Zheng and Dominic Brown and Pengbo Wang and Markus Hoehnerbach and Richard Cai and Julien Demouth and John D. Owens and Xia Hu and Song Han and Timmy Liu and Huizi Mao},
journal= {arXiv preprint arXiv:2512.12087},
year = {2026}
}