🤖 AI Summary
This study investigates whether iterative erasure methods can reliably quantify the number of directions in neural representations that encode specific concepts. By leveraging theoretical and experimental tools—including Gaussian population constructions, invertible shear transformations, QR decomposition, Moore–Penrose pseudoinverses, ridge regression, and Adam optimization—the work distinguishes between model-defined quantities (such as generative dimensionality and sufficient linear dimensionality) and process-dependent measures. The findings reveal that the stopping count and cumulative deletion rank obtained via iterative erasure lack invariance under information-preserving invertible reparameterizations, varying significantly with the choice of parameterization. This demonstrates that such metrics reflect properties of the measurement procedure rather than intrinsic characteristics of semantic dimensions, thereby exposing a fundamental limitation of iterative erasure as a measure of conceptual dimensionality.
📝 Abstract
How many directions does a neural representation use to encode a concept? A common answer repeatedly erases probe directions and reports the stopping count or cumulative removed rank. We show that both quantities can change under an information-preserving invertible reparameterization, so neither is intrinsically a concept dimension. We distinguish model-defined population quantities (generating dimension, sufficient linear dimension, and minimum guarding rank) from procedure-defined quantities such as stopping count and cumulative edit rank. In a population Gaussian construction, an invertible shear preserves the prediction problem and all three quantities, yet changes the cumulative Euclidean erasure count from one to two. The separation holds for Moore--Penrose ordinary least squares and every finite nonnegative ridge weight. For a two-output full-QR procedure matching our motivating video analysis, cumulative edit rank similarly changes from two to the ambient dimension four. Conversely, the complete cumulative metric-QR trajectory is affine-equivariant when its positive-definite metric, probe, regularizer, and tie-breaking are transported consistently; exact covariance is one corollary, not a canonical semantic metric. In a known-rank finite-sample Adam/QR calibration, identity mixing stops after one accepted update in all 20 large-sample runs, whereas each tested shear $a\in\{.5,.75,1,1.25,2\}$ accepts at least two updates in all 20 runs. Controlled reparameterizations of frozen V-JEPA2 features preserve rank-zero predictions yet alter later Euclidean trajectories under practical optimization. These visual contact experiments are stress tests, not estimates of contact dimension. Iterative erasure therefore returns a procedure-relative estimand jointly determined by representation geometry and the full measurement procedure, not a semantic dimension by itself.