Scalable Question-Centric Text-to-Image Evaluation: Reliable Ranking, Fine-Grained Diagnosis, and Cost-Aware Routing

2026-08-25

AI Summary

本文提出QC-T2I-Bench框架，通过将提示转换为具体问题并用Davidsonian场景图组织依赖关系，解决了现有文本到图像模型评估中难以区分不同模型优势的问题。

Abstract

Modern text-to-image (T2I) models often have similar total scores but different strengths, making practical selection difficult. Fine-grained benchmarks decompose prompts into questions, yet often return them to prompt scores and fixed categories, weakening attribution and ignoring complexity. Related requirements are also scored separately or as one total, obscuring basic versus compositional failure. We present QC-T2I-Bench, a question-centric framework that converts open prompts into attributed atomic questions and organizes their dependencies with Davidsonian Scene Graphs (DSGs). We use hierarchy-constrained question aggregation to exclude downstream questions after a prerequisite fails and to prevent simple and complex prompts from receiving the same total weight. We then use the DSG structure to measure joint success within prompts and compare repeated entities across prompts, separating basic realization failures from failures under additional requirements. We evaluate multiple open-source T2I models on English and Chinese prompts. The resulting question-level evidence supports reliable ranking and fine-grained diagnosis: joint completion falls from 80.7\% for components with two capabilities to 37.2\% for those with seven or more. Finally, we reuse the same records for training-free routing; our cost-aware router matches ERNIE's 89.51-point estimate with 21.3\% less GPU-s/MP.

Problem

• text-to-image models
• fine-grained benchmarks
• prompt decomposition
• complexity attribution
• basic versus compositional failure

Innovation

• Question-Centric Framework
• Davidsonian Scene Graphs (DSGs)
• Hierarchy-Constrained Question Aggregation
• Cost-Aware Routing