Benchmarking Layout-Guided Diffusion Models through Unified Semantic-Spatial Evaluation in Closed and Open Settings

2026-04-28

AI Summary

This work addresses the lack of an efficient and scalable evaluation benchmark that simultaneously accounts for semantic alignment and spatial consistency in layout-guided image generation, which has hindered fair model comparison and in-depth analysis. We propose the first dual-benchmark framework integrating a closed-set benchmark (C-Bench) and an open-set benchmark (O-Bench), covering both controllable and real-world scenarios, along with a unified semantic-spatial joint evaluation protocol. Leveraging this framework, we conduct a large-scale assessment of 319,086 images generated by six state-of-the-art layout-guided diffusion models, enabling fine-grained performance quantification, comprehensive ranking, and systematic insights into the strengths and limitations of current methods across varying prompt complexities and layout conditions.

Abstract

Evaluating layout-guided text-to-image generative models requires assessing both semantic alignment with textual prompts and spatial fidelity to prescribed layouts. Assessing layout alignment requires collecting fine-grained annotations, which is costly and labor-intensive. Consequently, current benchmarks rarely provide comprehensive layout evaluation and often remain limited in scale or coverage, making model comparison, ranking, and interpretation difficult. In this work, we introduce a closed-set benchmark (C-Bench) designed to isolate key generative capabilities while providing varying levels of complexity in both prompt structure and layout. To complement this controlled setting, we propose an open-set benchmark (O-Bench) that evaluates models using real-world prompts and layouts, offering a measure of semantic and spatial alignment in the wild. We further develop a unified evaluation protocol that combines semantic and spatial accuracy into a single score, ensuring consistent model ranking. Using our benchmarks, we conduct a large-scale evaluation of six state-of-the-art layout-guided diffusion models, totaling 319,086 generated and evaluated images. We establish a model ranking based on their overall performance and provide detailed breakdowns for text and layout alignment to enhance interpretability. Fine-grained analyses across scenarios and prompt complexities highlight the strengths and limitations of current models. Code is available at https://github.com/lparolari/cobench.

Problem

• layout-guided generation
• text-to-image models
• semantic alignment
• spatial fidelity
• benchmarking

Innovation

• layout-guided diffusion
• unified evaluation
• semantic-spatial alignment
• benchmarking
• text-to-image generation