Invertible Concept-based Explanations for CNN Models with Non-negative Concept Activation Vectors
Abstract
Convolutional neural network (CNN) models for computer vision are powerful but lack explainability in their most basic form. This deficiency remains a key challenge when applying CNNs in important domains. Recent work on explanations through feature importance of approximate linear models has moved from input-level features (pixels or segments) to features from mid-layer feature maps in the form of concept activation vectors (CAVs). CAVs contain concept-level information and could be learned via clustering. In this work, we rethink the ACE algorithm of Ghorbani et~al., proposing an alternative invertible concept-based explanation (ICE) framework to overcome its shortcomings. Based on the requirements of fidelity (approximate models to target models) and interpretability (being meaningful to people), we design measurements and evaluate a range of matrix factorization methods with our framework. We find that non-negative concept activation vectors (NCAVs) from non-negative matrix factorization provide superior performance in interpretability and fidelity based on computational and human subject experiments. Our framework provides both local and global concept-level explanations for pre-trained CNN models.
Cite
@article{arxiv.2006.15417,
title = {Invertible Concept-based Explanations for CNN Models with Non-negative Concept Activation Vectors},
author = {Ruihan Zhang and Prashan Madumal and Tim Miller and Krista A. Ehinger and Benjamin I. P. Rubinstein},
journal= {arXiv preprint arXiv:2006.15417},
year = {2021}
}