Bridging the Gap Between Saliency Prediction and Image Quality Assessment

2024-05-08

arXiv.org

AI Summary

This work investigates the intrinsic relationship between image quality assessment (IQA) and saliency prediction, examining whether IQA models implicitly leverage human visual saliency priors. To this end, we conduct cross-model performance analysis, construct human-perception-driven datasets, and perform large-scale benchmarking—thereby providing the first empirical evidence that mainstream IQA methods substantially benefit from explicit modeling of salient regions. Based on these findings, we introduce SACID, the first saliency-aware compressed image dataset, enabling IQA evaluation aligned with human visual cognition. We systematically evaluate both classical and deep IQA models on SACID and demonstrate that saliency-aware architectural designs improve prediction consistency and generalization across distortion types and compression levels. All code, annotations, and data are publicly released to foster reproducible research and advance cognitively grounded IQA methodologies.

Technology Category

• Computer Vision: Image and Video Retrieval
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Cognitive Modeling & Cognitive Systems: Computational Creativity

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Over the past few years, deep neural models have made considerable advances in image quality assessment (IQA). However, the underlying reasons for their success remain unclear, owing to the complex nature of deep neural networks. IQA aims to describe how the human visual system (HVS) works and to create its efficient approximations. On the other hand, Saliency Prediction task aims to emulate HVS via determining areas of visual interest. Thus, we believe that saliency plays a crucial role in human perception. In this work, we conduct an empirical study that reveals the relation between IQA and Saliency Prediction tasks, demonstrating that the former incorporates knowledge of the latter. Moreover, we introduce a novel SACID dataset of saliency-aware compressed images and conduct a large-scale comparison of classic and neural-based IQA methods. All supplementary code and data will be available at the time of publication.

Problem

• Explores the unclear success reasons of deep neural models in image quality assessment
• Investigates the relationship between image quality assessment and saliency prediction tasks
• Introduces a new SACID dataset for saliency-aware compressed image evaluation

Innovation

• Combines saliency prediction with IQA
• Introduces SACID dataset for evaluation
• Compares classic and neural IQA methods