OverLayBench: A Benchmark for Layout-to-Image Generation with Dense Overlaps

2025-09-23

AI Summary

This work addresses the degradation in layout-to-image generation quality caused by dense overlapping bounding boxes. It identifies two core challenges: large-scale spatial overlap and low semantic discriminability among overlapping instances. To mitigate evaluation bias in existing benchmarks (e.g., CLEVR-Layout), which favor low-overlap scenarios, we propose OverLayScore—a quantitative metric for overlap complexity—and introduce OverLayBench, the first high-overlap, balanced benchmark. We conduct the first systematic analysis of overlap’s impact on generation performance and fine-tune CreatiLayout-AM using amodal mask annotations, significantly enhancing modeling of overlapping regions. Experiments demonstrate substantial performance degradation of state-of-the-art methods under high OverLayScore conditions. Our work fills a critical gap in evaluating complex overlap scenarios, establishing a new standard and actionable directions for future research.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Computational Creativity
• Computer Vision: Language and Vision

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Despite steady progress in layout-to-image generation, current methods still struggle with layouts containing significant overlap between bounding boxes. We identify two primary challenges: (1) large overlapping regions and (2) overlapping instances with minimal semantic distinction. Through both qualitative examples and quantitative analysis, we demonstrate how these factors degrade generation quality. To systematically assess this issue, we introduce OverLayScore, a novel metric that quantifies the complexity of overlapping bounding boxes. Our analysis reveals that existing benchmarks are biased toward simpler cases with low OverLayScore values, limiting their effectiveness in evaluating model performance under more challenging conditions. To bridge this gap, we present OverLayBench, a new benchmark featuring high-quality annotations and a balanced distribution across different levels of OverLayScore. As an initial step toward improving performance on complex overlaps, we also propose CreatiLayout-AM, a model fine-tuned on a curated amodal mask dataset. Together, our contributions lay the groundwork for more robust layout-to-image generation under realistic and challenging scenarios. Project link: https://mlpc-ucsd.github.io/OverLayBench.

Problem

• Addressing layout-to-image generation challenges with overlapping bounding boxes
• Quantifying overlap complexity to evaluate model performance limitations
• Creating benchmark for robust image generation under challenging overlap scenarios

Innovation

• Introducing OverLayScore metric for overlap complexity
• Creating OverLayBench benchmark with balanced score distribution
• Proposing CreatiLayout-AM model fine-tuned on amodal masks