Interpretability without actionability: mechanistic methods cannot correct language model errors despite near-perfect internal representations
Abstract
Language models encode task-relevant knowledge in internal representations that far exceeds their output performance, but whether mechanistic interpretability methods can bridge this knowledge-action gap has not been systematically tested. We compared four mechanistic interpretability methods -- concept bottleneck steering (Steerling-8B), sparse autoencoder feature steering, logit lens with activation patching, and linear probing with truthfulness separator vector steering (Qwen 2.5 7B Instruct) -- for correcting false-negative triage errors using 400 physician-adjudicated clinical vignettes (144 hazards, 256 benign). Linear probes discriminated hazardous from benign cases with 98.2% AUROC, yet the model's output sensitivity was only 45.1%, a 53-percentage-point knowledge-action gap. Concept bottleneck steering corrected 20% of missed hazards but disrupted 53% of correct detections, indistinguishable from random perturbation (p=0.84). SAE feature steering produced zero effect despite 3,695 significant features. TSV steering at high strength corrected 24% of missed hazards while disrupting 6% of correct detections, but left 76% of errors uncorrected. Current mechanistic interpretability methods cannot reliably translate internal knowledge into corrected outputs, with implications for AI safety frameworks that assume interpretability enables effective error correction.
Cite
@article{arxiv.2603.18353,
title = {Interpretability without actionability: mechanistic methods cannot correct language model errors despite near-perfect internal representations},
author = {Sanjay Basu and Sadiq Y. Patel and Parth Sheth and Bhairavi Muralidharan and Namrata Elamaran and Aakriti Kinra and John Morgan and Rajaie Batniji},
journal= {arXiv preprint arXiv:2603.18353},
year = {2026}
}
Comments
27 pages, 5 figures, 10 tables. Code available at https://github.com/sanjaybasu/interpretability-triage