Reasoning emerges from constrained inference manifolds in large language models

2026-05-02

AI Summary

Current evaluation methods conflate the task performance of large language models with their intrinsic reasoning capabilities, overlooking the dynamic and evolving nature of these systems. This work conceptualizes reasoning as a process wherein internal representations self-organize into low-dimensional manifolds embedded in high-dimensional spaces. It establishes, for the first time, that effective reasoning must satisfy three structural conditions—expressivity, manifold compression, and preservation of information volume—from geometric and informational perspectives. Building upon representational geometry, manifold learning, and dynamical systems theory, we introduce a unified, label-free diagnostic framework that relies solely on internal model dynamics. Empirical results demonstrate that reasoning quality is governed by the structural properties of these intrinsic dynamics, and our approach effectively identifies pathological reasoning behaviors beyond conventional performance metrics.

Abstract

Reasoning in large language models is predominantly evaluated through labeled benchmarks, conflating task performance with the quality of internal inference. Here we study reasoning as an intrinsic dynamical process by examining the evolution of internal representations during inference. We find that inference-time dynamics consistently self-organize into low-dimensional manifolds embedded within high-dimensional representation spaces. we find that such geometric compression, although pervasive, is not sufficient for stable or reliable reasoning. Instead, effective reasoning dynamics emerge within a constrained structural regime characterized by three conditions: adequate representational expressivity, spontaneous manifold compression, and preservation of non-degenerate information volume within the compressed subspace. Models outside this regime exhibit characteristic pathological inference dynamics. Based on these insights, we introduce a unified, label-free diagnostic computed solely from internal dynamics. These findings suggest that reasoning in LLMs is fundamentally governed by geometric and informational constraints, offering a complementary framework to benchmark-centric assessment.

Problem

• reasoning
• large language models
• inference dynamics
• representation manifolds
• geometric constraints

Innovation

• inference dynamics
• manifold compression
• representational geometry
• information volume
• label-free diagnostic