English

Iterative Erasure Count Is Not an Affine-Invariant Concept Dimension

Machine Learning 2026-08-11 v1 Computer Vision and Pattern Recognition Machine Learning

Abstract

How many directions does a neural representation use to encode a concept? A common answer repeatedly erases probe directions and reports the stopping count or cumulative removed rank. We show that both quantities can change under an information-preserving invertible reparameterization, so neither is intrinsically a concept dimension. We distinguish model-defined population quantities (generating dimension, sufficient linear dimension, and minimum guarding rank) from procedure-defined quantities such as stopping count and cumulative edit rank. In a population Gaussian construction, an invertible shear preserves the prediction problem and all three quantities, yet changes the cumulative Euclidean erasure count from one to two. The separation holds for Moore--Penrose ordinary least squares and every finite nonnegative ridge weight. For a two-output full-QR procedure matching our motivating video analysis, cumulative edit rank similarly changes from two to the ambient dimension four. Conversely, the complete cumulative metric-QR trajectory is affine-equivariant when its positive-definite metric, probe, regularizer, and tie-breaking are transported consistently; exact covariance is one corollary, not a canonical semantic metric. In a known-rank finite-sample Adam/QR calibration, identity mixing stops after one accepted update in all 20 large-sample runs, whereas each tested shear a{.5,.75,1,1.25,2}a\in\{.5,.75,1,1.25,2\} accepts at least two updates in all 20 runs. Controlled reparameterizations of frozen V-JEPA2 features preserve rank-zero predictions yet alter later Euclidean trajectories under practical optimization. These visual contact experiments are stress tests, not estimates of contact dimension. Iterative erasure therefore returns a procedure-relative estimand jointly determined by representation geometry and the full measurement procedure, not a semantic dimension by itself.

Keywords

Cite

@article{arxiv.2608.10566,
  title  = {Iterative Erasure Count Is Not an Affine-Invariant Concept Dimension},
  author = {Tingan Jin and Shuhang Dong and Haosong Li and Chung-Hsien Chou},
  journal= {arXiv preprint arXiv:2608.10566},
  year   = {2026}
}

Comments

15 pages, 3 figures. Code and results included as ancillary files. Tingan Jin, Shuhang Dong, and Haosong Li contributed equally