The Geometric Reasoner: Manifold-Informed Latent Foresight Search for Long-Context Reasoning

2026-01-25

AI Summary

Existing approaches to long-chain reasoning struggle to balance computational efficiency with trajectory diversity. This work proposes a training-free inference framework that enhances long-context reasoning through manifold-informed latent look-ahead search. By integrating lightweight look-ahead estimation, soft geometric regularization, chunk-wise KV cache resetting, and a latent anchor scoring mechanism, the method enables smooth and diverse exploration of reasoning trajectories while maintaining linear memory growth. Evaluated on the Qwen3-8B model, the approach achieves up to a 13-percentage-point improvement in the area under the Pass@$k$ curve (AUC), with only a 1.1–1.3× increase in computational overhead.

Technology Category

• Search and Optimization: Learning to Search
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Scaling test-time compute enhances long chain-of-thought (CoT) reasoning, yet existing approaches face a fundamental trade-off between computational cost and coverage quality: either incurring high training expense or yielding redundant trajectories. We introduce The Geometric Reasoner (TGR), a training-free framework that performs manifold-informed latent foresight search under strict memory bounds. At each chunk boundary, TGR scores candidate latent anchors via a lightweight look-ahead estimate combined with soft geometric regularizers that encourage smooth trajectories and diverse exploration. Chunk-wise KV cache resets keep memory linear in chunk length. On challenging math and code benchmarks, TGR improves robust trajectory coverage, measured by the area under the Pass@$k$ curve (AUC), by up to 13 points on Qwen3-8B, with negligible overhead of about 1.1--1.3 times.

Problem

• long-context reasoning
• chain-of-thought
• computational cost
• trajectory coverage
• test-time scaling

Innovation

• manifold-informed search
• latent foresight
• training-free reasoning
• KV cache reset
• geometric regularizers