Scaling Spatial Intelligence with Multimodal Foundation Models

2025-11-17

AI Summary

Current multimodal foundation models exhibit significant limitations in spatial intelligence—encompassing geometric reasoning, 3D understanding, and spatial relational modeling. To address this, we introduce the SenseNova-SI series, built upon architectures including Qwen3-VL, InternVL3, and Bagel, and trained via large-scale supervised fine-tuning under a unified spatial intelligence paradigm spanning visual understanding and generation. We propose a systematic taxonomy of spatial capabilities, curate 8 million high-quality, diverse spatial data samples, and incorporate data debiasing and structured annotation strategies. Experiments demonstrate state-of-the-art performance on spatial intelligence benchmarks—including VSI-Bench, MMSI, and MindCube—while preserving strong general multimodal competence. Notably, we report the first empirical observation of diversity-driven emergent spatial generalization and provide preliminary validation of spatial chain-of-thought reasoning. All models and datasets are publicly released.

Technology Category

• Computer Vision: Multi-modal Vision
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Despite remarkable progress, multimodal foundation models still exhibit surprising deficiencies in spatial intelligence. In this work, we explore scaling up multimodal foundation models to cultivate spatial intelligence within the SenseNova-SI family, built upon established multimodal foundations including visual understanding models (i.e., Qwen3-VL and InternVL3) and unified understanding and generation models (i.e., Bagel). We take a principled approach to constructing high-performing and robust spatial intelligence by systematically curating SenseNova-SI-8M: eight million diverse data samples under a rigorous taxonomy of spatial capabilities. SenseNova-SI demonstrates unprecedented performance across a broad range of spatial intelligence benchmarks: 68.7% on VSI-Bench, 43.3% on MMSI, 85.6% on MindCube, 54.6% on ViewSpatial, and 50.1% on SITE, while maintaining strong general multimodal understanding (e.g., 84.9% on MMBench-En). More importantly, we analyze the impact of data scaling, discuss early signs of emergent generalization capabilities enabled by diverse data training, analyze the risk of overfitting and language shortcuts, present a preliminary study on spatial chain-of-thought reasoning, and validate the potential downstream application. SenseNova-SI is an ongoing project, and this report will be updated continuously. All newly trained multimodal foundation models are publicly released to facilitate further research in this direction.

Problem

• Addressing spatial intelligence deficiencies in multimodal foundation models
• Scaling up models to cultivate spatial reasoning capabilities systematically
• Developing robust spatial intelligence through diverse data curation

Innovation

• Scaling multimodal models for spatial intelligence
• Creating eight million diverse spatial data samples
• Achieving top performance across spatial benchmarks