Quality Assessment for AI Generated Images with Instruction Tuning

2024-05-12

AI Summary

Existing image quality assessment (IQA) methods struggle to jointly model multi-dimensional human preferences—particularly quality, realism, and semantic consistency—for AI-generated images (AIGIs). Method: We introduce AIGCIQA2023+, the first large-scale, multi-dimensional subjective IQA database specifically for AIGIs, featuring fine-grained human annotations across quality, realism, and consistency dimensions. Building upon it, we propose MINT-IQA—a novel, interpretable, multi-perspective IQA framework leveraging instruction-tuned vision-language large models to jointly predict preference scores and generate natural-language feedback. Contribution/Results: MINT-IQA achieves state-of-the-art performance on AIGI preference assessment and matches top-performing models on conventional IQA benchmarks (LIVE, KADID). Both the AIGCIQA2023+ database and the MINT-IQA code are publicly released.

Technology Category

• Humans and AI: Learning Human Values and Preferences
• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Artificial Intelligence Generated Content (AIGC) has grown rapidly in recent years, among which AI-based image generation has gained widespread attention due to its efficient and imaginative image creation ability. However, AI-generated Images (AIGIs) may not satisfy human preferences due to their unique distortions, which highlights the necessity to understand and evaluate human preferences for AIGIs. To this end, in this paper, we first establish a novel Image Quality Assessment (IQA) database for AIGIs, termed AIGCIQA2023+, which provides human visual preference scores and detailed preference explanations from three perspectives including quality, authenticity, and correspondence. Then, based on the constructed AIGCIQA2023+ database, this paper presents a MINT-IQA model to evaluate and explain human preferences for AIGIs from Multi-perspectives with INstruction Tuning. Specifically, the MINT-IQA model first learn and evaluate human preferences for AI-generated Images from multi-perspectives, then via the vision-language instruction tuning strategy, MINT-IQA attains powerful understanding and explanation ability for human visual preference on AIGIs, which can be used for feedback to further improve the assessment capabilities. Extensive experimental results demonstrate that the proposed MINT-IQA model achieves state-of-the-art performance in understanding and evaluating human visual preferences for AIGIs, and the proposed model also achieves competing results on traditional IQA tasks compared with state-of-the-art IQA models. The AIGCIQA2023+ database and MINT-IQA model are available at: https://github.com/IntMeGroup/MINT-IQA.

Problem

• AI Generated Images
• Image Quality
• Perceived Realism

Innovation

• AIGCIQA2023+
• MINT-IQA
• AI-generated image quality assessment