Zero-Ablation Overstates Register Content Dependence in DINO Vision Transformers

Zero-ablation – replacing token activations with zero vectors – is widely used to probe token function in vision transformers. Register zeroing in DINOv2+registers and DINOv3 produces large drops (up to $-36.6$,pp classification, $-30.9$,pp segmentation), suggesting registers are functionally indispensable. However, three replacement controls – mean-substitution, noise-substitution, and cross-image register-shuffling – preserve performance across classification, correspondence, and segmentation, remaining within $\sim 1$,pp of the unmodified baseline. Per-patch cosine similarity shows these replacements genuinely perturb internal representations, while zeroing causes disproportionately large perturbations, consistent with why it alone degrades tasks. We conclude that zero-ablation overstates dependence on exact register content. In the frozen-feature evaluations we test, performance depends on plausible register-like activations rather than on exact image-specific values. Registers nevertheless buffer dense features from $\mathtt{\text{[CLS]}}$ dependence and are associated with compressed patch geometry. These findings, including the replacement-control results, replicate at ViT-B scale.