Zero-ablation -- replacing token activations with zero vectors -- is widely used to probe token function in vision transformers. Register zeroing in DINOv2+registers and DINOv3 produces large drops (up to −36.6\,pp classification, −30.9\,pp segmentation), suggesting registers are functionally indispensable. However, three replacement controls -- mean-substitution, noise-substitution, and cross-image register-shuffling -- preserve performance across classification, correspondence, and segmentation, remaining within ∼1\,pp of the unmodified baseline. Per-patch cosine similarity shows these replacements genuinely perturb internal representations, while zeroing causes disproportionately large perturbations, consistent with why it alone degrades tasks. We conclude that zero-ablation overstates dependence on exact register content. In the frozen-feature evaluations we test, performance depends on plausible register-like activations rather than on exact image-specific values. Registers nevertheless buffer dense features from \texttt{[CLS]} dependence and are associated with compressed patch geometry. These findings, including the replacement-control results, replicate at ViT-B scale.
Cite
@article{arxiv.2604.14433,
title = {Zero-Ablation Overstates Register Content Dependence in DINO Vision Transformers},
author = {Felipe Parodi and Jordan Matelsky and Melanie Segado},
journal= {arXiv preprint arXiv:2604.14433},
year = {2026}
}
Comments
12 pages, 10 figures, to be published in CVPR 2026 HOW Vision Interpretability Workshop Proceedings