OrdShap: Feature Position Importance for Sequential Black-Box Models
Abstract
Sequential deep learning models excel in domains with temporal or sequential dependencies, but their complexity necessitates post-hoc feature attribution methods for understanding their predictions. While existing techniques quantify feature importance, they inherently assume fixed feature ordering - conflating the effects of (1) feature values and (2) their positions within input sequences. To address this gap, we introduce OrdShap, a novel attribution method that disentangles these effects by quantifying how a model's predictions change in response to permuting feature position. We establish a game-theoretic connection between OrdShap and Sanchez-Berganti\~nos values, providing a theoretically grounded approach to position-sensitive attribution. Empirical results from health, natural language, and synthetic datasets highlight OrdShap's effectiveness in capturing feature value and feature position attributions, and provide deeper insight into model behavior.
Cite
@article{arxiv.2507.11855,
title = {OrdShap: Feature Position Importance for Sequential Black-Box Models},
author = {Davin Hill and Brian L. Hill and Aria Masoomi and Vijay S. Nori and Robert E. Tillman and Jennifer Dy},
journal= {arXiv preprint arXiv:2507.11855},
year = {2025}
}
Comments
Advances in Neural Information Processing Systems, Volume 38 (NeurIPS 2025)