Large Language Models (LLMs) can generate plausible free text self-explanations to justify their answers. However, these natural language explanations may not accurately reflect the model's actual reasoning process, pinpointing a lack of faithfulness. Existing faithfulness evaluation methods rely primarily on behavioral tests or computational block analysis without examining the semantic content of internal neural representations. This paper proposes NeuroFaith, a flexible framework that measures the faithfulness of LLM free text self-explanation by identifying key concepts within explanations and mechanistically testing whether these concepts actually influence the model's predictions. We show the versatility of NeuroFaith across 2-hop reasoning and classification tasks. Additionally, we develop a linear faithfulness probe based on NeuroFaith to detect unfaithful self-explanations from representation space and improve faithfulness through steering. NeuroFaith provides a principled approach to evaluating and enhancing the faithfulness of LLM free text self-explanations, addressing critical needs for trustworthy AI systems.
@article{arxiv.2506.09277,
title = {NeuroFaith: Evaluating LLM Self-Explanation Faithfulness via Internal Representation Alignment},
author = {Milan Bhan and Jean-Noel Vittaut and Nicolas Chesneau and Sarath Chandar and Marie-Jeanne Lesot},
journal= {arXiv preprint arXiv:2506.09277},
year = {2026}
}