TextGround4M: A Prompt-Aligned Dataset for Layout-Aware Text Rendering

2026-04-27

AI Summary

Current text-to-image generation models struggle to accurately render textual content specified in prompts along with its spatial layout, particularly in multi-segment, structured scenes, due to a lack of aligned data and appropriate evaluation metrics. To address this, this work introduces TextGround4M, a large-scale dataset comprising four million prompt–image pairs, each annotated with bounding boxes precisely aligning textual fragments from the prompt to their visual counterparts. Furthermore, the authors propose a lightweight training strategy that requires no architectural modifications, leveraging layout-aware textual fragment tokens to enable fine-grained supervision. Experimental results demonstrate that this approach significantly outperforms strong baselines in terms of text fidelity, spatial accuracy, and prompt adherence, thereby validating the critical role of fine-grained layout supervision in enhancing controllable text-to-image synthesis.

Technology Category

• Natural Language Processing: Language Grounding & Multi-modal NLP
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Large Vision Models

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Despite recent advances in text-to-image generation, models still struggle to accurately render prompt-specified text with correct spatial layout -- especially in multi-span, structured settings. This challenge is driven not only by the lack of datasets that align prompts with the exact text and layout expected in the image, but also by the absence of effective metrics for evaluating layout quality. To address these issues, we introduce TextGround4M, a large-scale dataset of over 4 million prompt-image pairs, each annotated with span-level text grounded in the prompt and corresponding bounding boxes. This enables fine-grained supervision for layout-aware, prompt-grounded text rendering. Building on this, we propose a lightweight training strategy for autoregressive T2I models that appends layout-aware span tokens during training, without altering model architecture or inference behavior. We further construct a benchmark with stratified layout complexity to evaluate both open-source and proprietary models in a zero-shot setting. In addition, we introduce two layout-aware metrics to address the long-standing lack of spatial evaluation in text rendering. Our results show that models trained on TextGround4M outperform strong baselines in text fidelity, spatial accuracy, and prompt consistency, highlighting the importance of fine-grained layout supervision for grounded T2I generation.

Problem

• text-to-image generation
• spatial layout
• prompt alignment
• text rendering
• layout evaluation

Innovation

• layout-aware text rendering
• prompt-aligned dataset
• span-level grounding
• lightweight training strategy
• spatial evaluation metrics