🤖 AI Summary
This work addresses the limitation of existing behavioral benchmarks, which capture only model outputs and fail to reveal internal mechanisms—particularly the structural discontinuities induced by curriculum shifts. The authors propose a training-free mechanistic probe that leverages spectral graph theory to analyze the algebraic connectivity (λ₂), smoothness, and spectral entropy of attention graphs, thereby constructing a “spectral fingerprint” that characterizes the causal topological structure underlying model evolution. They identify and name a novel phenomenon—“Passive-Triggered Connectivity Collapse” (PTCC)—demonstrating how curriculum changes disrupt syntactic sensitivity. A topology-based auditing framework is established and validated across 12 models and 10 languages, confirming fingerprint stability. Furthermore, sparse compensatory heads are localized to pinpoint failure mechanisms; guided activation of these heads restores approximately 38% of lost information flow, and the study reveals that topological structure is primarily governed by subword token density rather than language family.
📝 Abstract
Behavioral benchmarks tell us \textit{what} a model does, but not \textit{how}. We introduce a training-free mechanistic probe using attention-graph spectra. Treating each layer as a token graph, we compute algebraic connectivity ($\lambda_2$), smoothness, and spectral entropy. Across 12 models and 10 languages, these measures yield stable ``spectral fingerprints''that expose discontinuities missed by standard evaluation. We report four results. (1) Models undergoing specific curriculum transitions (e.g., code-to-chat) show an English-only, syntax-triggered connectivity failure on non-canonical constructions, reaching $\Delta\lambda_2 \approx -0.76$. We term this scar \textit{Passive-Triggered Connectivity Collapse} (PTCC). Analysis of the Phi lineage reveals that PTCC appears and resolves across developmental stages, implicating brittle curriculum shifts rather than synthetic data per se. (2) PTCC reflects a specialization trade-off: strengthened formal routing at the expense of stylistic flexibility. (3) We identify four recurrent processing strategies; simple frozen-threshold rules enable perfect forensic identification across lineages. (4) Mechanistically, PTCC localizes to a sparse Layer 2 ``compensatory patch''of heads that fails under syntactic stress; activation steering can partially restore connectivity, recovering $\approx 38\%$ of lost information flow. Finally, dominant topological regimes track tokenization density more than language identity, suggesting ``healthy''geometry varies systematically across scripts. Overall, attention-graph spectra provide a practical tool for auditing and training-regime verification.