Implicit Geometry of Next-token Prediction: From Language Sparsity Patterns to Model Representations

2024-08-27

arXiv.org

AI Summary

This work investigates how next-token prediction (NTP) implicitly shapes the geometric structure of language model representations. We formulate NTP as a sparse soft-label classification task and theoretically establish its equivalence to nuclear-norm-regularized low-rank optimization in the logit space. This analysis reveals that NTP inherently favors learning logits with a “sparse + low-rank” structure. Empirically, we identify a “subspace collapse” phenomenon: contexts sharing the same set of successor tokens automatically cluster into specific low-dimensional subspaces in the embedding space. We validate these geometric regularities on both synthetic data and small-scale real corpora. Our findings provide an interpretable geometric perspective and a formal theoretical framework for understanding how NTP encodes linguistic statistical patterns—linking representational geometry, optimization bias, and emergent linguistic structure in autoregressive language modeling.

Technology Category

• Natural Language Processing: Learning & Optimization for NLP
• Machine Learning: Structured Learning
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Next-token prediction (NTP) over large text corpora has become the go-to paradigm to train large language models. Yet, it remains unclear how NTP influences the mapping of linguistic patterns to geometric properties of the resulting model representations. We frame training of large language models as soft-label classification over sparse probabilistic label vectors, coupled with an analytical approximation that allows unrestricted generation of context embeddings. This approach links NTP training to rank-constrained, nuclear-norm regularized optimization in the logit domain, offering a framework for analyzing the geometry of word and context embeddings. In large embedding spaces, we find that NTP implicitly favors learning logits with a sparse plus low-rank structure. While the sparse component captures the co-occurrence frequency of context-word pairs, the orthogonal low-rank component, which becomes dominant as training progresses, depends solely on the sparsity pattern of the co-occurrence matrix. Consequently, when projected onto an appropriate subspace, representations of contexts that are followed by the same set of next-tokens collapse, a phenomenon we term subspace-collapse. We validate our findings on synthetic and small-scale real language datasets. Finally, we outline potential research directions aimed at deepening the understanding of NTP's influence on the learning of linguistic patterns and regularities.

Problem

• Analyzing next-token prediction's geometric influence.
• Exploring sparse plus low-rank logit structures.
• Investigating subspace-collapse in context representations.

Innovation

• Soft-label classification with sparse vectors
• Nuclear-norm regularized optimization in logit domain
• Subspace-collapse in context embeddings