GeoAnchor: Collaborative Reasoning via Latent Decomposition for 3D Spatial Understanding

2026-07-15

AI Summary

This work addresses the challenge that existing multimodal large language models struggle to accurately represent continuous geometric information when reasoning about 3D spatial relationships from 2D images, primarily due to their reliance on symbolic text. To overcome this limitation, the authors propose GeoAnchor, a novel framework that decouples 3D spatial information into three complementary latent representations—position, orientation, and geometry—and dynamically recombines them within a structured space to integrate local evidence with global context. Leveraging a text–latent interleaved reasoning architecture and a co-training strategy, the model achieves interpretable and task-adaptive spatial reasoning. Extensive experiments demonstrate that GeoAnchor significantly outperforms current state-of-the-art methods across diverse and complex 3D reasoning tasks, confirming its effectiveness and strong generalization capability.

Abstract

Although multimodal large language models (MLLMs) have achieved remarkable progress, understanding 3D spatial relationships from 2D images remains a critical challenge. Existing methods primarily rely on symbolic text tokens, which inherently lack the fidelity to represent continuous geometric information. While recent methods use latent representations to enhance reasoning, relying on a single latent type cannot adapt to the diversity of spatial tasks, leading to misalignment in complex geometric scenarios. To address these limitations, we propose GeoAnchor, an interleaved text-latent reasoning framework. GeoAnchor decomposes 3D spatial information into three complementary components: position latents for object grounding, direction latents for relational orientation, and geometry latents for scene structure. These components are recombined in a structured space to construct local evidence while capturing global context, enabling dynamic and interpretable reasoning. Furthermore, we introduce a collaborative training strategy that guides the model from local spatial perception to comprehensive 3D understanding. Extensive experiments on diverse and complex 3D reasoning tasks demonstrate that GeoAnchor outperforms the state of the art, validating its effectiveness and generalization capabilities.

Problem

• 3D spatial understanding
• multimodal large language models
• latent representation
• geometric reasoning
• spatial relationships

Innovation

• latent decomposition
• 3D spatial reasoning
• multimodal large language models
• structured representation
• collaborative training