English

Learning to Explain: Supervised Token Attribution from Transformer Attention Patterns

Computation and Language 2026-01-22 v2 Machine Learning

Abstract

Explainable AI (XAI) has become critical as transformer-based models are deployed in high-stakes applications including healthcare, legal systems, and financial services, where opacity hinders trust and accountability. Transformers self-attention mechanisms have proven valuable for model interpretability, with attention weights successfully used to understand model focus and behavior (Xu et al., 2015); (Wiegreffe and Pinter, 2019). However, existing attention-based explanation methods rely on manually defined aggregation strategies and fixed attribution rules (Abnar and Zuidema, 2020a); (Chefer et al., 2021), while model-agnostic approaches (LIME, SHAP) treat the model as a black box and incur significant computational costs through input perturbation. We introduce Explanation Network (ExpNet), a lightweight neural network that learns an explicit mapping from transformer attention patterns to token-level importance scores. Unlike prior methods, ExpNet discovers optimal attention feature combinations automatically rather than relying on predetermined rules. We evaluate ExpNet in a challenging cross-task setting and benchmark it against a broad spectrum of model-agnostic methods and attention-based techniques spanning four methodological families.

Keywords

Cite

@article{arxiv.2601.14112,
  title  = {Learning to Explain: Supervised Token Attribution from Transformer Attention Patterns},
  author = {George Mihaila},
  journal= {arXiv preprint arXiv:2601.14112},
  year   = {2026}
}
R2 v1 2026-07-01T09:12:41.901Z