Sparse Concept Channels in Frozen 3D CT Vision Encoders

Farhad Nooralahzadeh; Lea Bogensperger; Christian Bluethgen; Michael Krauthammer

Sparse Concept Channels in Frozen 3D CT Vision Encoders

Computer Vision and Pattern Recognition 2026-07-23 v1 Artificial Intelligence

Authors: Farhad Nooralahzadeh , Lea Bogensperger , Christian Bluethgen , Michael Krauthammer

Abstract

Large vision-language models are becoming increasingly dominant in 3D medical image interpretation, but we rarely know which internal units encode clinical findings or where that information lives in the representation. We first study this on a 3D chest vision-language model (Pillar-0) by probing its frozen vision embeddings. We show that (i) each radiological finding is encoded by a sparse set of ~10 vision-encoder channels that match full-feature classification performance and far exceed a zero-shot text prompting; (ii) turning off the channels tied to one finding, that finding's score collapses while unrelated labels stay stable; and (iii) the same sparse probe replicates on an architecturally unrelated 3D abdominal VLM (Merlin) suggesting a general property of frozen medical encoders. Our training-free concept channel probe (CCP) method, paired with a corpus-derived report template, outperforms published CT-CHAT on clinical efficacy and NLG metrics (F1 0.549 vs. 0.184; BLEU 0.483 vs. 0.373) at 22x lower latency. Our results provide a clear, reproducible characterization of how frozen medical encoders represent findings, demonstrating direct applicability across models.

Keywords

concept channels 3D CT vision encoders vision-language models interpretability sparse probing medical imaging

Cite

@article{arxiv.2607.20993,
  title  = {Sparse Concept Channels in Frozen 3D CT Vision Encoders},
  author = {Farhad Nooralahzadeh and Lea Bogensperger and Christian Bluethgen and Michael Krauthammer},
  journal= {arXiv preprint arXiv:2607.20993},
  year   = {2026}
}