On the Limitations of Cross-Lingual Consistency in Multilingual Text-to-image Generation

2026-08-11

AI Summary

This work addresses the significant cross-lingual performance disparities and language-specific effects in multilingual text-to-image (T2I) generation, which have lacked systematic evaluation due to the absence of a comprehensive benchmark. To bridge this gap, we introduce LingT2I—the first holistic evaluation benchmark for multilingual T2I models—encompassing 10 languages and 33,000 prompts. Through multilingual prompt engineering and cross-lingual consistency assessment, combined with both qualitative and quantitative analyses, LingT2I systematically evaluates model performance in content generation and text rendering. Our study reveals substantial variations in generation quality across languages, identifies systematic influences of linguistic structure and cultural context on outputs, and provides the first empirical evidence of linguistic inequality in T2I systems, thereby laying the groundwork for developing fairer and more robust multilingual T2I models.

Abstract

Text-to-image (T2I) generation has achieved remarkable progress in recent years. However, existing research has largely focused on English-only settings, leaving cross-lingual performance gaps and language-specific effects insufficiently explored. To fill this gap, we introduce LingT2I, a benchmark covering 10 widely used languages with 33K prompts, designed to evaluate cross-lingual effects in both content generation and text rendering. Building on this benchmark, we conduct a comprehensive cross-lingual analysis, uncovering linguistic inequality and language-dependent trade-offs across evaluation dimensions. Beyond quantitative evaluation, we further reveal a range of language-dependent generation patterns, highlighting how linguistic factors and their corresponding cultural contexts systematically impact model outputs. Our benchmark and analysis provide a foundation for studying cross-lingual behavior in T2I generation and facilitate the development of more robust and inclusive models. Code and dataset are available at https://github.com/RISys-Lab/LingT2I.

Problem

• text-to-image generation
• cross-lingual consistency
• multilingual
• language-specific effects
• linguistic inequality

Innovation

• cross-lingual consistency
• multilingual text-to-image generation
• LingT2I benchmark
• language-dependent generation
• linguistic inequality