Spatial Chain-of-Thought: Bridging Understanding and Generation Models for Spatial Reasoning Generation

2026-02-12

AI Summary

Existing diffusion models exhibit limited performance on complex spatial reasoning tasks, while approaches integrating multimodal large language models (MLLMs) often suffer from high computational costs or loss of spatial information due to reliance on purely textual prompts. This work proposes Spatial Chain-of-Thought (SCoT), a plug-and-play framework that, for the first time, introduces structured spatial instructions with interleaved coordinate-text representations into diffusion model training. By leveraging an MLLM as a spatial planner to generate layout guidance, SCoT efficiently transfers the MLLM’s spatial reasoning capabilities without requiring joint training. The method significantly outperforms existing baselines on image generation and editing benchmarks, achieving end-to-end, high-fidelity image synthesis with strong spatial understanding.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

While diffusion models have shown exceptional capabilities in aesthetic image synthesis, they often struggle with complex spatial understanding and reasoning. Existing approaches resort to Multimodal Large Language Models (MLLMs) to enhance this capability. However, they either incur high computational costs through joint training or suffer from spatial information loss when relying solely on textual prompts. To alleviate these limitations, we propose a Spatial Chain-of-Thought (SCoT) framework, a plug-and-play approach that effectively bridges the reasoning capabilities of MLLMs with the generative power of diffusion models. Specifically, we first enhance the diffusion model's layout awareness by training it on an interleaved text-coordinate instruction format. We then leverage state-of-the-art MLLMs as planners to generate comprehensive layout plans, transferring their spatial planning capabilities directly to the generation process. Extensive experiments demonstrate that our method achieves state-of-the-art performance on image generation benchmarks and significantly outperforms baselines on complex reasoning tasks, while also showing strong efficacy in image editing scenarios.

Problem

• spatial reasoning
• diffusion models
• multimodal large language models
• spatial understanding
• image generation

Innovation

• Spatial Chain-of-Thought
• Diffusion Models
• Multimodal Large Language Models
• Spatial Reasoning
• Layout-aware Generation