A Multi-Domain Benchmark for Detecting AI-Generated Text-Rich Images from GPT-Image-2

2026-06-17

AI Summary

Existing benchmarks for detecting AI-generated images inadequately cover text- and layout-centric image-text compositions, limiting their ability to address the realism challenges posed by multimodal generative models. This work introduces TextGenBench, the first benchmark specifically designed for detecting GPT-Image-2–generated text-dense images, comprising 8,602 samples across six categories: commercial posters, infographics, academic posters, receipts, tables, and UI screenshots. Under a zero-shot setting, the study systematically evaluates multiple detectors in terms of performance, domain dependency, and robustness to post-processing operations such as JPEG compression. Results reveal that current methods exhibit significant performance disparities across categories and are highly sensitive to compression artifacts. Although vision-language models show promise, they remain limited in understanding structured visual formats.

Abstract

Text-rich images often contain privacy-sensitive, transactional, or decision-relevant information. As recent multimodal image generation models become increasingly capable of synthesizing realistic textual content and structured visual designs, detecting AI-generated text-rich images has become an important challenge for digital trust and content authenticity. Existing benchmarks, however, largely focus on object-centric images and provide limited coverage of scenarios where textual semantics and layout organization are central. In this paper, we introduce a multi-domain benchmark for detecting text-rich images generated by OpenAI's GPT Image 2. The benchmark contains 8,602 images across six representative categories: commercial posters, infographics, academic posters, receipts, tables, and UI screenshots. Using this benchmark, we evaluate five representative AI-generated image detectors in a zero-shot setting and analyze their overall, category-wise, and post-processing robustness. Our results show that detector performance is highly domain-dependent: methods that perform well in some categories often fail on others, and even the strongest conventional detector exhibits severe sensitivity to JPEG compression. We further conduct an exploratory evaluation with a multimodal vision-language model, revealing both its promise and its limitations on structured formats. These findings highlight the need for text- and layout-aware detection methods for modern AI-generated images. Our dataset is released at XXX.

Problem

• AI-generated text-rich images
• image detection
• digital trust
• content authenticity
• multimodal generation

Innovation

• text-rich image detection
• multimodal benchmark
• AI-generated image
• layout-aware detection
• zero-shot evaluation