SemRF: A Semantic Reference Frame for Residual-Stream Dynamics in Language Models

2026-06-30

AI Summary

This work addresses semantic measurement drift in residual stream analysis of language models, caused by misalignment between embedding and unembedding coordinates, which obscures true computational dynamics. To resolve this, the paper introduces the Semantic Reference Frame (SemRF), which anchors semantic coordinates via fixed reference points to disentangle semantic representations from residual evolution, enabling stable cross-layer tracking of semantic trajectories. The method integrates pseudo-inverse binding, bi-invertibility constraints, semantic Voronoi diagrams, and action optimization within margin-augmented relaxation tubes to construct low-curvature, piecewise-linear compressed trajectories. Under strict interface error bounds, the framework guarantees trajectory uniqueness and minimizes semantic degrees of freedom while revealing local knowledge density, thereby establishing a conditional link between semantic dynamics in residual streams and parameter efficiency.

Abstract

Residual-stream analysis asks how language-model computation evolves across depth, but intermediate decoding requires comparable readout coordinates across layers. If embedding anchors and unembedding readout disagree on the chosen span, apparent motion may reflect measurement drift rather than computation. We introduce \emph{Semantic Reference Frames} (SemRF), an anchor-based formalism separating semantic measurement from residual dynamics. A SemRF fixes anchors and measures states against them. Pseudo-inverse tying gives exact synchronization; under restricted bi-invertibility, SemRF yields stable semantic-basis coordinates, distortion bounds, and near-identity changes. With the frame fixed, residual computation becomes a depthwise semantic trajectory. The anchors induce a semantic Voronoi diagram: distance, or evidence such as logits, assigns each layer to a coarse cell, while coordinates retain within-cell motion and margins. We define layerwise steps, contribution profiles, and imbalance diagnostics, then use the Voronoi trace to define a margin-relaxed tube. The canonical trace is the minimum-action path inside this tube; when nonempty with positive quadratic weight, it is unique and obeys a discrete spline equation away from active constraints. Excess action controls step, curvature, and profile mismatch. Low curvature implies piecewise-linear compressibility and local knowledge density: lower trace complexity means fewer semantic knots. Through the parameter-to-trajectory map, this gives a conditional link to parameter efficiency: among admissible settings fitting data, lower-action and lower-complexity traces use fewer semantic degrees of freedom. The guarantees require controlled interface error and small projection residual under explicit tube constraints.

Problem

• residual-stream dynamics
• semantic reference frame
• layerwise computation
• measurement drift
• semantic coordinates

Innovation

• Semantic Reference Frame
• Residual Stream
• Minimum-Action Path
• Voronoi Diagram
• Parameter Efficiency