Structure Before Collapse: Transient semantic geometry in next-token prediction

2026-06-25

AI Summary

This work demonstrates that, despite theoretical expectations that training with purely one-hot labels should yield symmetric representations devoid of semantic structure, language models spontaneously develop meaningful semantic geometry during early training stages. By designing synthetic tasks in which inputs share latent semantic factors yet map to distinct one-hot labels, the study employs contextual embedding analysis, dynamic tracking of Gram matrices, and geometric modeling of representations to reveal a transient phase where embeddings cluster according to shared attributes before collapsing into symmetric configurations. The authors propose an enhanced unconstrained feature model to characterize this transient geometry, providing the first empirical validation of the ephemeral existence of semantic structure prior to convergence. These findings indicate that gradient descent can implicitly capture semantic categories before reaching the optimization endpoint, offering new insights into the mechanisms underlying representation learning in language models.

Abstract

Neural Collapse predicts that balanced one-hot classification pushes model representations to be equally far from each other; a symmetric configuration that depends only on the output label and ignores any semantic similarity in the inputs. This creates a puzzle: next-token prediction language models are trained predominantly (as context length increases) with one-hot labels: the same context is very unlikely to appear twice in training with different labels. However, they clearly learn latent structural features. That is, despite the one-hot training regime, a language model's contextual embeddings represent the fact that the next word in ''Mary broke the ___'' is likely to be filled by tokens in the latent classes of a) medium-sized, b) rigid, c) inanimate nouns. How does gradient descent find such categorical semantic structure when co-occurrence statistics collapse to one-hot sparsity, eliminating any shared next-tokens among different contexts? To investigate this tension we identify three synthetic controlled settings where inputs have latent semantic factors but are mapped to distinct one-hot labels. We find that semantic geometry emerges early in training, and that representations cluster by shared attributes despite receiving no explicit supervision to do so. This structure is transient: with sufficient capacity and time, the model eventually reaches the predicted symmetric state where all representations are equally separated. We study this phase transition through Gram matrix analysis and propose a preliminary modification to the commonly used unconstrained features model to capture the emergent semantic geometry.

Problem

• next-token prediction
• neural collapse
• semantic geometry
• one-hot labels
• latent structure

Innovation

• Neural Collapse
• semantic geometry
• next-token prediction
• transient representation
• Gram matrix analysis