While recent years have witnessed the emergence of various explainable methods in machine learning, to what degree the explanations really represent the reasoning process behind the model prediction -- namely, the faithfulness of explanation -- is still an open problem. One commonly used way to measure faithfulness is \textit{erasure-based} criteria. Though conceptually simple, erasure-based criterion could inevitably introduce biases and artifacts. We propose a new methodology to evaluate the faithfulness of explanations from the \textit{counterfactual reasoning} perspective: the model should produce substantially different outputs for the original input and its corresponding counterfactual edited on a faithful feature. Specially, we introduce two algorithms to find the proper counterfactuals in both discrete and continuous scenarios and then use the acquired counterfactuals to measure faithfulness. Empirical results on several datasets show that compared with existing metrics, our proposed counterfactual evaluation method can achieve top correlation with the ground truth under diffe
@article{arxiv.2109.01962,
title = {Counterfactual Evaluation for Explainable AI},
author = {Yingqiang Ge and Shuchang Liu and Zelong Li and Shuyuan Xu and Shijie Geng and Yunqi Li and Juntao Tan and Fei Sun and Yongfeng Zhang},
journal= {arXiv preprint arXiv:2109.01962},
year = {2021}
}