Although state-space models (SSMs) have demonstrated strong performance on long-sequence benchmarks, most research has emphasized predictive accuracy rather than interpretability. In this work, we present the first systematic kernel interpretability study of the diagonalized state-space model (S4D) trained on a real-world task (vulnerability detection in source code). Through time and frequency domain analysis of the S4D kernel, we show that the long-range modeling capability of S4D varies significantly under different model architectures, affecting model performance. For instance, we show that the depending on the architecture, S4D kernel can behave as low-pass, band-pass or high-pass filter. The insights from our analysis can guide future work in designing better S4D-based models.
Cite
@article{arxiv.2601.13048,
title = {Analysis of Long Range Dependency Understanding in State Space Models},
author = {Srividya Ravikumar and Abhinav Anand and Shweta Verma and Mira Mezini},
journal= {arXiv preprint arXiv:2601.13048},
year = {2026}
}