LLMs as High-Dimensional Nonlinear Autoregressive Models with Attention: Training, Alignment and Inference

2026-01-31

AI Summary

This work addresses the lack of a unified mathematical characterization of the underlying computational mechanisms in large language models (LLMs), as existing research has predominantly focused on architectures and training pipelines. The paper proposes formalizing LLMs as high-dimensional nonlinear autoregressive systems equipped with attention mechanisms, revealing for the first time the intrinsic bilinear–Softmax–linear structure of self-attention within a single coherent framework. This formulation seamlessly integrates pretraining, alignment methods—including RLHF, DPO, RSFT, and RLVR—and inference-time generation. By establishing an equation-level reference framework, the study provides a theoretical foundation for analyzing emergent phenomena such as hallucination, sycophancy, and in-context learning, thereby enabling principled interpretation and systematic investigation of model behavior.

Technology Category

• Natural Language Processing: (Large) Language Models
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Large Vision Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Large language models for search

Abstract

Large language models (LLMs) based on transformer architectures are typically described through collections of architectural components and training procedures, obscuring their underlying computational structure. This review article provides a concise mathematical reference for researchers seeking an explicit, equation-level description of LLM training, alignment, and generation. We formulate LLMs as high-dimensional nonlinear autoregressive models with attention-based dependencies. The framework encompasses pretraining via next-token prediction, alignment methods such as reinforcement learning from human feedback (RLHF), direct preference optimization (DPO), rejection sampling fine-tuning (RSFT), and reinforcement learning from verifiable rewards (RLVR), as well as autoregressive generation during inference. Self-attention emerges naturally as a repeated bilinear--softmax--linear composition, yielding highly expressive sequence models. This formulation enables principled analysis of alignment-induced behaviors (including sycophancy), inference-time phenomena (such as hallucination, in-context learning, chain-of-thought prompting, and retrieval-augmented generation), and extensions like continual learning, while serving as a concise reference for interpretation and further theoretical development.

Problem

• large language models
• autoregressive models
• attention mechanism
• alignment
• inference

Innovation

• nonlinear autoregressive models
• attention mechanism
• alignment theory
• self-attention formulation
• inference phenomena