Spatial Causal Prediction in Video

2026-03-04

AI Summary

This work addresses the limited capacity of existing video understanding models to perform causal inference over unobserved spatiotemporal states, particularly in predicting past or future spatial causal dynamics. To this end, we introduce a novel task termed Spatial Causal Prediction (SCP) and present SCP-Bench, a benchmark comprising 2,500 question-answer pairs. We systematically evaluate 23 state-of-the-art models by integrating multi-view video analysis, causal direction modeling, perception enhancement, and reasoning-guided strategies to assess their spatiotemporal causal reasoning capabilities. Experimental results demonstrate that current models significantly underperform humans in temporal extrapolation and causal grounding, while our proposed approach effectively enhances spatial causal reasoning performance, thereby highlighting both the challenge and research value of the SCP task.

Technology Category

• Computer Vision: Video Understanding & Activity Analysis
• Reasoning under Uncertainty: Causality
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding
• Web Mining and Content Analysis: Large pretrained models with web data
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

Spatial reasoning, the ability to understand spatial relations, causality, and dynamic evolution, is central to human intelligence and essential for real-world applications such as autonomous driving and robotics. Existing studies, however, primarily assess models on visible spatio-temporal understanding, overlooking their ability to infer unseen past or future spatial states. In this work, we introduce Spatial Causal Prediction (SCP), a new task paradigm that challenges models to reason beyond observation and predict spatial causal outcomes. We further construct SCP-Bench, a benchmark comprising 2,500 QA pairs across 1,181 videos spanning diverse viewpoints, scenes, and causal directions, to support systematic evaluation. Through comprehensive experiments on {23} state-of-the-art models, we reveal substantial gaps between human and model performance, limited temporal extrapolation, and weak causal grounding. We further analyze key factors influencing performance and propose perception-enhancement and reasoning-guided strategies toward advancing spatial causal intelligence. The project page is https://guangstrip.github.io/SCP-Bench.

Problem

• Spatial Causal Prediction
• spatial reasoning
• causal inference
• video understanding
• temporal extrapolation

Innovation

• Spatial Causal Prediction
• SCP-Bench
• causal reasoning
• temporal extrapolation
• spatial intelligence