Image-Difficulty-Aware Evaluation of Super-Resolution Models

2025-09-30

AI Summary

Conventional super-resolution (SR) evaluation relies on average metrics, failing to reveal performance disparities across images of varying difficulty and unable to capture structural artifacts prevalent in challenging samples. Method: We propose a difficulty-aware evaluation paradigm featuring two interpretable, image-level difficulty quantification metrics: the High-frequency Spectral Index (HSI), derived from frequency-domain analysis, and the Rotation-Invariant Edge Index (RIEI), based on structural edge detection. These metrics enable difficulty stratification, forming the basis of an objective evaluation framework complemented by subjective comparative assessment for fine-grained model performance decomposition. Contribution/Results: Experiments demonstrate that our approach effectively discriminates between SR models with similar average scores but markedly different real-world behaviors. It significantly enhances evaluation granularity and discriminative power across multiple benchmarks, establishing a new standard for precise, difficulty-sensitive SR model assessment.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Evaluation and Analysis
• Search and Optimization: Evaluation and Analysis

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Security and Privacy: Large-scale security measurements

Abstract

Image super-resolution models are commonly evaluated by average scores (over some benchmark test sets), which fail to reflect the performance of these models on images of varying difficulty and that some models generate artifacts on certain difficult images, which is not reflected by the average scores. We propose difficulty-aware performance evaluation procedures to better differentiate between SISR models that produce visually different results on some images but yield close average performance scores over the entire test set. In particular, we propose two image-difficulty measures, the high-frequency index and rotation-invariant edge index, to predict those test images, where a model would yield significantly better visual results over another model, and an evaluation method where these visual differences are reflected on objective measures. Experimental results demonstrate the effectiveness of the proposed image-difficulty measures and evaluation methodology.

Problem

• Evaluating super-resolution models beyond average benchmark scores
• Identifying difficult images causing artifacts in model outputs
• Proposing difficulty-aware metrics to differentiate visually distinct results

Innovation

• Proposes high-frequency index for image difficulty
• Introduces rotation-invariant edge index measure
• Develops difficulty-aware evaluation methodology for super-resolution