GeoSense: Internalizing Geometric Necessity Perception for Multimodal Reasoning

2026-03-11

AI Summary

This work addresses the inefficiency of multimodal large language models in spatial reasoning, which often stems from their indiscriminate injection of geometric information, leading to computational redundancy. The authors propose a novel approach that introduces a dedicated geometric input channel alongside a spatial-aware supervised fine-tuning dataset, endowing the model— for the first time—with an intrinsic capability to assess the necessity of geometric cues and activate them only when required. This method achieves significant performance gains across multiple spatial reasoning benchmarks while preserving the model’s original 2D visual reasoning capabilities, thereby effectively balancing accuracy and computational efficiency.

Technology Category

• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Multi-modal Vision

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Advancing towards artificial superintelligence requires rich and intelligent perceptual capabilities. A critical frontier in this pursuit is overcoming the limited spatial understanding of Multimodal Large Language Models (MLLMs), where geometry information is essential. Existing methods often address this by rigidly injecting geometric signals into every input, while ignoring their necessity and adding computation overhead. Contrary to this paradigm, our framework endows the model with an awareness of perceptual insufficiency, empowering it to autonomously engage geometric features in reasoning when 2D cues are deemed insufficient. To achieve this, we first introduce an independent geometry input channel to the model architecture and conduct alignment training, enabling the effective utilization of geometric features. Subsequently, to endow the model with perceptual awareness, we curate a dedicated spatial-aware supervised fine-tuning dataset. This serves to activate the model's latent internal cues, empowering it to autonomously determine the necessity of geometric information. Experiments across multiple spatial reasoning benchmarks validate this approach, demonstrating significant spatial gains without compromising 2D visual reasoning capabilities, offering a path toward more robust, efficient and self-aware multi-modal intelligence.

Problem

• spatial reasoning
• geometric perception
• multimodal large language models
• perceptual awareness

Innovation

• geometric reasoning
• multimodal large language models
• perceptual awareness
• spatial reasoning
• self-adaptive perception