谨慎的谨慎：概念基准可解释 AI 中的场景表示是否足够

可解释人工智能 (XAI) 旨在帮助揭示 AI 模型内部表征中的缺陷，但人们是否能从其解释中得出正确的结论？ Specifically, do they recognize an AI's inability to distinguish between relevant and irrelevant features? In the present study, a simulated AI classified images of railway trespassers as dangerous or not. To explain which features it has used, other images from the dataset were shown that activate the AI in a similar way. These concept images varied in three relevant features (i.e., a person's distance to the tracks, direction, and action) and in an irrelevant feature (i.e., scene background). When the AI uses a feature in its decision, this feature is retained in the concept images, otherwise the images randomize over it (e.g., same distance, varied backgrounds). Participants rated the AI more favorably when it retained relevant features. For the irrelevant feature, they did not mind in general, and sometimes even preferred it to be retained. This suggests that people may not recognize it when an AI model relies on irrelevant features to make its decisions.