Multi-Level Explanations for Generative Language Models
Abstract
Despite the increasing use of large language models (LLMs) for context-grounded tasks like summarization and question-answering, understanding what makes an LLM produce a certain response is challenging. We propose Multi-Level Explanations for Generative Language Models (MExGen), a technique to provide explanations for context-grounded text generation. MExGen assigns scores to parts of the context to quantify their influence on the model's output. It extends attribution methods like LIME and SHAP to LLMs used in context-grounded tasks where (1) inference cost is high, (2) input text is long, and (3) the output is text. We conduct a systematic evaluation, both automated and human, of perturbation-based attribution methods for summarization and question answering. The results show that our framework can provide more faithful explanations of generated output than available alternatives, including LLM self-explanations. We open-source code for MExGen as part of the ICX360 toolkit: https://githubcom/IBM/ICX360.
Cite
@article{arxiv.2403.14459,
title = {Multi-Level Explanations for Generative Language Models},
author = {Lucas Monteiro Paes and Dennis Wei and Hyo Jin Do and Hendrik Strobelt and Ronny Luss and Amit Dhurandhar and Manish Nagireddy and Karthikeyan Natesan Ramamurthy and Prasanna Sattigeri and Werner Geyer and Soumya Ghosh},
journal= {arXiv preprint arXiv:2403.14459},
year = {2025}
}
Comments
Accepted as an oral presentation at ACL 2025. Code available at https://github.com/IBM/ICX360