Towards Spatial Supersensing in the Wild

2026-07-15

AI Summary

Existing spatial super-perception models are predominantly evaluated on synthetic videos and struggle to effectively capture the continuity and complexity inherent in real-world, long-duration, and diverse scenarios. To address this limitation, this work introduces VSI-Super-Wild, a large-scale benchmark comprising over four hours of real-world long-form video across eight scene categories. Inspired by structured human experience from cognitive science, the authors design an evaluation suite based on human-validated question-answer pairs. They propose a diagnostic framework that systematically assesses spatiotemporal consistency across agent-object-environment triadic world states, identifying four failure modes of current spatial world models. The study reveals significant performance degradation in existing models under long temporal spans and complex state configurations, highlighting their inability to construct coherent unified spatial representations and delineating critical challenges for future research.

Abstract

Humans can efficiently parse continuous sensory streams, from hours to years, scaffolding an internal world model that grounds spatial reasoning and prediction. To mimic this capacity, spatial supersensing challenges multimodal models to move beyond linguistic understanding toward true world modeling. However, their benchmark relies on synthetic long videos, formed by concatenating random short clips, and is mostly limited to household scenes, leaving real-world continuity and diversity underexplored. To address the gap, we introduce $\textbf{VSI-Super-Wild}$, a large-scale benchmark for evaluating spatial supersensing over long temporal horizons in diverse in-the-wild scenes. Notably, inspired by cognitive studies on how humans structure experience, we systematically probe the full triad of world state: the agent (observer), objects (scene items), and the environment (places and global layout). In total, VSI-Super-Wild contains $\textbf{6,980}$ human-verified question-answer pairs derived from $\textbf{442}$ real-world videos spanning 8 scene categories, including long-form recordings exceeding 4 hours. Results on VSI-Super-Wild expose a fundamental disconnect: despite advances in static image understanding, models consistently fail at tasks that require coherent world-state tracking over time. We characterize how performance degrades with world-state complexity and temporal horizon, and diagnose four failure modes: spatial collapse, semantic shortcuts, insufficient update, and instance confusion. This taxonomy reveals that models lack mechanisms to bind objects, agents, and environments into a unified spatial world model, a fundamental gap that defines the path forward for spatial supersensing.

Problem

• spatial supersensing
• world modeling
• temporal continuity
• in-the-wild scenes
• world-state tracking

Innovation

• spatial supersensing
• world modeling
• long-form video understanding
• multimodal reasoning
• VSI-Super-Wild