Spatial4D-Bench: A Versatile 4D Spatial Intelligence Benchmark

2025-12-31

arXiv.org

AI Summary

This work evaluates whether multimodal large language models (MLLMs) approach human-level performance in four-dimensional spatial intelligence—the ability to perceive and reason about objects’ motion and transformation over time. To this end, we introduce the first large-scale, structured benchmark for 4D spatial intelligence, encompassing six cognitive categories, 18 distinct tasks, and approximately 40,000 question-answer pairs, thereby overcoming the limitations of existing benchmarks in scale and diversity. Systematic evaluation across multiple open- and closed-source MLLMs reveals significant deficiencies in tasks such as path planning, action recognition, and reasoning about physical plausibility, highlighting a pronounced gap between current models and human spatial cognition capabilities.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Large language models for search

Abstract

4D spatial intelligence involves perceiving and processing how objects move or change over time. Humans naturally possess 4D spatial intelligence, supporting a broad spectrum of spatial reasoning abilities. To what extent can Multimodal Large Language Models (MLLMs) achieve human-level 4D spatial intelligence? In this work, we present Spatial4D-Bench, a versatile 4D spatial intelligence benchmark designed to comprehensively assess the 4D spatial reasoning abilities of MLLMs. Unlike existing spatial intelligence benchmarks that are often small-scale or limited in diversity, Spatial4D-Bench provides a large-scale, multi-task evaluation benchmark consisting of ~40,000 question-answer pairs covering 18 well-defined tasks. We systematically organize these tasks into six cognitive categories: object understanding, scene understanding, spatial relationship understanding, spatiotemporal relationship understanding, spatial reasoning and spatiotemporal reasoning. Spatial4D-Bench thereby offers a structured and comprehensive benchmark for evaluating the spatial cognition abilities of MLLMs, covering a broad spectrum of tasks that parallel the versatility of human spatial intelligence. We benchmark various state-of-the-art open-source and proprietary MLLMs on Spatial4D-Bench and reveal their substantial limitations in a wide variety of 4D spatial reasoning aspects, such as route plan, action recognition, and physical plausibility reasoning. We hope that the findings provided in this work offer valuable insights to the community and that our benchmark can facilitate the development of more capable MLLMs toward human-level 4D spatial intelligence. More resources can be found on our project page.

Problem

• 4D spatial intelligence
• Multimodal Large Language Models
• spatial reasoning
• benchmark
• spatiotemporal reasoning

Innovation

• 4D spatial intelligence
• multimodal large language models
• spatiotemporal reasoning
• benchmark
• spatial cognition