🤖 AI Summary
This work investigates the dynamic evolution of Transformer representations across layers, moving beyond static analyses of encoded content. It introduces a probe-free geometric framework that treats forward propagation as a discrete trajectory on a high-dimensional representation manifold and directly computes five geometric metrics—trajectory length, curvature, semantic convergence index, inter-layer cosine similarity, and representational stability—in the original embedding space. The study systematically uncovers the geometric signatures of semantic convergence, reasoning complexity, and ambiguity resolution: semantically related prompts exhibit pronounced convergence in middle-to-late layers, reasoning tasks yield trajectories with higher curvature, and ambiguous tokens induce trajectory bifurcations. Furthermore, it identifies a universal three-stage representational evolution pattern across architectures, substantially enhancing the interpretability of model mechanisms.
📝 Abstract
Understanding how transformer representations evolve across layers, not merely what they encode, remains an open problem in mechanistic interpretability. We recast the transformer forward pass as a discrete population trajectory through a high-dimensional representation manifold, drawing on geometric tools from computational neuroscience. Rather than probing for pre-specified features, we characterize trajectory geometry using five metrics computed directly in the ambient space: trajectory length, curvature, a semantic convergence index, layerwise cosine similarity, and representational stability. Across three model families (GPT-2, TinyLlama, Qwen2.5) and five controlled prompt families, we report four findings. First, semantically related prompts converge significantly in middle-to-late layers (peak CI 0.41--0.58, p<0.001, Mann-Whitney U), consistent with attractor-like dynamics. Second, reasoning tasks produce trajectories of greater curvature than lexical variations (0.71--0.83 rad vs. 0.27--0.31 rad), suggesting curvature encodes computational complexity. Third, ambiguous tokens exhibit trajectory bifurcation with up to 5.6x representational separation by the final layer, absent in unambiguous controls. Fourth, layerwise cosine similarity reveals a universal three-phase structure: encoding, elaboration, and output preparation, consistent across all three architectures. All four effects vanish under shuffled-layer and random-embedding controls. We release a fully open-source, model-agnostic pipeline and argue that trajectory geometry constitutes a principled, probe-free lens for mechanistic interpretability.