This paper introduces a rigorous mathematical framework for neural network explainability, and more broadly for the explainability of equivariant operators called Group Equivariant Operators (GEOs) based on Group Equivariant Non-Expansive Operators (GENEOs) transformations. The central concept involves quantifying the distance between GEOs by measuring the non-commutativity of specific diagrams. Additionally, the paper proposes a definition of interpretability of GEOs according to a complexity measure that can be defined according to each user preferences. Moreover, we explore the formal properties of this framework and show how it can be applied in classical machine learning scenarios, like image classification with convolutional neural networks.
Cite
@article{arxiv.2503.01942,
title = {Mathematical Foundation of Interpretable Equivariant Surrogate Models},
author = {Jacopo Joy Colombini and Filippo Bonchi and Francesco Giannini and Fosca Giannotti and Roberto Pellungrini and Patrizio Frosini},
journal= {arXiv preprint arXiv:2503.01942},
year = {2025}
}