VQualA 2025 Challenge on Image Super-Resolution Generated Content Quality Assessment: Methods and Results

2025-09-08

AI Summary

Existing objective image quality assessment (IQA) metrics lack effectiveness in evaluating perceptual quality of generative super-resolution (SR) outputs—particularly artifacts unique to GANs and diffusion models. Method: We introduce ISRGen-QA, the first high-quality benchmark dataset specifically designed for modern generative SR methods, and organize an international competition to foster innovation. Ground-truth quality labels are established via rigorous subjective experiments; mainstream objective metrics are systematically evaluated; and deep learning–based distortion modeling is encouraged. Crucially, we propose the first artifact-aware IQA paradigm explicitly tailored to generative SR characteristics. Contribution/Results: This work fills a critical research gap in SR quality evaluation in the generative-model era. The top-performing method achieves state-of-the-art (SOTA) performance on ISRGen-QA, demonstrating significantly improved assessment accuracy. The competition attracted 108 participants, with four teams submitting valid solutions.

Technology Category

• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Natural Language Processing: Generation
• Knowledge Representation and Reasoning: Qualitative Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Web data quality in the era of algorithmically-generated content
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

This paper presents the ISRGC-Q Challenge, built upon the Image Super-Resolution Generated Content Quality Assessment (ISRGen-QA) dataset, and organized as part of the Visual Quality Assessment (VQualA) Competition at the ICCV 2025 Workshops. Unlike existing Super-Resolution Image Quality Assessment (SR-IQA) datasets, ISRGen-QA places a greater emphasis on SR images generated by the latest generative approaches, including Generative Adversarial Networks (GANs) and diffusion models. The primary goal of this challenge is to analyze the unique artifacts introduced by modern super-resolution techniques and to evaluate their perceptual quality effectively. A total of 108 participants registered for the challenge, with 4 teams submitting valid solutions and fact sheets for the final testing phase. These submissions demonstrated state-of-the-art (SOTA) performance on the ISRGen-QA dataset. The project is publicly available at: https://github.com/Lighting-YXLI/ISRGen-QA.

Problem

• Assessing perceptual quality of super-resolution images
• Analyzing artifacts from generative SR methods
• Evaluating GAN and diffusion model outputs

Innovation

• Challenge on super-resolution generated content assessment
• Focuses on GANs and diffusion models artifacts
• Evaluates perceptual quality with SOTA methods