Do Large Language Models Play Six Degrees of Separation? Measuring Topological Compression in Long-Context Manifolds

2026-08-18

AI Summary

本文通过分析大语言模型隐藏状态流形的动态几何，证明其深层潜在空间自然组织成小世界网络，解决了多跳推理机制理解问题。

Abstract

Large Language Models (LLMs) demonstrate remarkable multi-hop reasoning capabilities over long contexts, yet the internal mechanisms enabling these distant cognitive leaps remain poorly understood. Traditional attention-based interpretability often fails to capture true semantic proximity due to routing artifacts like attention sinks. In this paper, we bypass attention weights to directly analyze the dynamic geometry of the hidden state manifold, proving that deep LLM latent spaces natively organize into Small-World networks. By sparsifying the continuous similarity matrices of long-context representations into unweighted graphs, we trace the connectivity between highly disjoint semantic anchors across two distinct architectures. Our findings reveal a sharp topological phase transition: while early syntactic layers remain entirely fractured, deep reasoning layers abruptly compress massive conceptual distances into highly navigable pathways strictly bounded by the "Six Degrees of Separation" limit (=< 6 semantic hops). Furthermore, we demonstrate the practical efficacy of this framework by applying it to zero-shot hallucination detection within Retrieval-Augmented Generation (RAG) using the RAGognize dataset. We show that factually grounded generations maintain structural integrity with their source context (approximately 3 hops), whereas hallucinations induce severe topological collapse. Ultimately, this work mathematically formalizes how transformers execute abstract reasoning and provides a novel, strictly geometric signature for evaluating factual reliability.

Problem

• Large Language Models
• Multi-hop Reasoning
• Semantic Proximity
• Attention Sinks
• Hidden State Manifold

Innovation

• Small-World networks
• Topological Compression
• Long-Context Manifolds
• Semantic Proximity
• Zero-shot Hallucination Detection