Processing long-form videos with Video Large Language Models (Video-LLMs) is computationally prohibitive. Current efficiency methods often compromise fine-grained perception through irreversible information disposal or inhibit long-range temporal modeling via rigid, predefined sparse patterns. This paper introduces AdaSpark, an adaptive sparsity framework designed to address these limitations. AdaSpark first partitions video inputs into 3D spatio-temporal cubes. It then employs two co-designed, context-aware components: (1) Adaptive Cube-Selective Attention (AdaS-Attn), which adaptively selects a subset of relevant video cubes to attend for each query token, and (2) Adaptive Token-Selective FFN (AdaS-FFN), which selectively processes only the most salient tokens within each cube. An entropy-based (Top-p) selection mechanism adaptively allocates computational resources based on input complexity. Experiments demonstrate that AdaSpark significantly reduces computational load by up to 57% FLOPs while maintaining comparable performance to dense models and preserving fine-grained, long-range dependencies, as validated on challenging hour-scale video benchmarks.
@article{arxiv.2604.08077,
title = {AdaSpark: Adaptive Sparsity for Efficient Long-Video Understanding},
author = {Handong Li and Zikang Liu and Longteng Guo and Tongtian Yue and Yepeng Tang and Xinxin Zhu and Chuanyang Zheng and Ziming Wang and Zhibin Wang and Jun Song and Cheng Yu and Bo Zheng and Jing Liu},
journal= {arXiv preprint arXiv:2604.08077},
year = {2026}
}