Spatial-Conditioned Reasoning in Long-Egocentric Videos

2026-01-26

AI Summary

This work addresses the challenges posed by long-horizon first-person videos, where viewpoint drift and the absence of persistent geometric context severely hinder visual navigation and spatial reasoning. To enhance the spatially conditioned reasoning capabilities of off-the-shelf vision-language models without altering their architecture or inference pipeline, the authors propose an input-level inductive bias that explicitly fuses RGB frames with depth maps as spatial signals. To support evaluation of navigation-oriented spatial queries, they introduce Sanpo-D, a fine-grained annotated dataset. Experimental results demonstrate that this approach significantly improves performance on safety-critical tasks such as pedestrian detection and obstacle recognition, while also revealing a fundamental trade-off between general-purpose capabilities and spatial specialization in vision-language models.

Technology Category

• Computer Vision: Language and Vision
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Natural Language Processing: Safety and Robustness

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Long-horizon egocentric video presents significant challenges for visual navigation due to viewpoint drift and the absence of persistent geometric context. Although recent vision-language models perform well on image and short-video reasoning, their spatial reasoning capability in long egocentric sequences remains limited. In this work, we study how explicit spatial signals influence VLM-based video understanding without modifying model architectures or inference procedures. We introduce Sanpo-D, a fine-grained re-annotation of the Google Sanpo dataset, and benchmark multiple VLMs on navigation-oriented spatial queries. To examine input-level inductive bias, we further fuse depth maps with RGB frames and evaluate their impact on spatial reasoning. Our results reveal a trade-off between general-purpose accuracy and spatial specialization, showing that depth-aware and spatially grounded representations can improve performance on safety-critical tasks such as pedestrian and obstruction detection.

Problem

• spatial reasoning
• egocentric video
• visual navigation
• viewpoint drift
• geometric context

Innovation

• spatial reasoning
• egocentric video
• vision-language models
• depth fusion
• navigation