ArchSIBench: Benchmarking the Architectural Spatial Intelligence of Vision-Language Models

2026-05-20

AI Summary

This study addresses the lack of systematic evaluation of existing vision-language models in high-level spatial cognition relevant to architectural intelligence—such as layout understanding, circulation organization, and functional zoning. To bridge this gap, the authors propose ArchSIBench, the first multidimensional benchmark specifically designed for architectural spatial intelligence. Grounded in perspectives from architecture, cognitive science, and psychology, ArchSIBench encompasses 17 fine-grained tasks across five dimensions: perception, reasoning, navigation, transformation, and configuration, built upon 3,000 expert-annotated question-answer pairs. The benchmark also introduces human baselines with architectural expertise for comparative analysis. Experimental results reveal that state-of-the-art models significantly underperform professionals in spatial transformation and configuration reasoning, with only a few advanced models approaching the performance of non-expert humans, thereby underscoring ArchSIBench’s critical role in addressing the evaluation gap in high-order spatial cognition.

Abstract

Architectural spatial intelligence, the ability to recognize and infer architectural space, is fundamental to tasks such as robot navigation, embodied interaction, and 3D scene understanding and generation. Although extensive research has evaluated the basic spatial skills of Vision-Language Models (VLMs) such as relative orientation, distance comparison, and object counting, these tasks cover only the most elementary levels of spatial cognition and largely overlook higher-level cognition of architectural space, including layout understanding, circulation patterns, and functional zoning. In this work, we present ArchSIBench, a Benchmark for Architectural Spatial Intelligence based on the perspectives from architecture, cognitive science, and psychology. ArchSIBench covers five core dimensions: perception, reasoning, navigation, transformation, and configuration, comprising 17 fine-grained subtasks. Through careful manual annotation by experts with architectural backgrounds, we construct 3,000 question-answer pairs to enable comprehensive evaluation of architectural spatial intelligence. Based on ArchSIBench, we evaluate various VLMs and find that the architectural spatial intelligence of most models shows significant differences from human baselines; additionally, models exhibit substantial variability across capability dimensions. Some state-of-the-art models can approach the level of human evaluators without architectural training. However, a clear gap remains compared to human evaluators with architectural training, particularly in spatial transformation and configuration reasoning. We believe that ArchSIBench will provide important insights and systematic resources for measuring and advancing the architectural spatial intelligence of VLMs. The dataset and code are available at https://huggingface.co/datasets/ArchSIBench/ArchSIBench.

Problem

• Architectural Spatial Intelligence
• Vision-Language Models
• Spatial Cognition
• Benchmarking
• 3D Scene Understanding

Innovation

• Architectural Spatial Intelligence
• Vision-Language Models
• Benchmarking
• Spatial Reasoning
• 3D Scene Understanding