Understanding the decision-making process of machine learning models provides valuable insights into the task, the data, and the reasons behind a model's failures. In this work, we propose a method that performs inherently interpretable predictions through the instance-wise sparsification of input images. To align the sparsification with human perception, we learn the masking in the space of semantically meaningful pixel regions rather than on pixel-level. Additionally, we introduce an explicit way to dynamically determine the required level of sparsity for each instance. We show empirically on semi-synthetic and natural image datasets that our inherently interpretable classifier produces more meaningful, human-understandable predictions than state-of-the-art benchmarks.
@article{arxiv.2505.06003,
title = {From Pixels to Perception: Interpretable Predictions via Instance-wise Grouped Feature Selection},
author = {Moritz Vandenhirtz and Julia E. Vogt},
journal= {arXiv preprint arXiv:2505.06003},
year = {2025}
}