MDS-VQA: Model-Informed Data Selection for Video Quality Assessment

2026-03-12

AI Summary

This work addresses the disconnect between model design and dataset construction in existing video quality assessment (VQA) research, where efficient data selection mechanisms targeting model weaknesses are lacking. The authors propose a model-guided data selection approach that uniquely integrates failure prediction with semantic diversity. Specifically, they employ a ranking-based failure predictor to estimate sample difficulty and leverage deep semantic features to quantify content diversity, then apply a greedy algorithm to balance these two criteria for selecting high-value unlabeled videos. Using only a 5% curated subset for fine-tuning, the method improves the average SRCC from 0.651 to 0.722 and achieves top performance on the gMAD benchmark, demonstrating substantially enhanced generalization capability.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Computer Vision: Diffusion Models for Vision
• Search and Optimization: Learning to Search

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Learning-based video quality assessment (VQA) has advanced rapidly, yet progress is increasingly constrained by a disconnect between model design and dataset curation. Model-centric approaches often iterate on fixed benchmarks, while data-centric efforts collect new human labels without systematically targeting the weaknesses of existing VQA models. Here, we describe MDS-VQA, a model-informed data selection mechanism for curating unlabeled videos that are both difficult for the base VQA model and diverse in content. Difficulty is estimated by a failure predictor trained with a ranking objective, and diversity is measured using deep semantic video features, with a greedy procedure balancing the two under a constrained labeling budget. Experiments across multiple VQA datasets and models demonstrate that MDS-VQA identifies diverse, challenging samples that are particularly informative for active fine-tuning. With only a 5% selected subset per target domain, the fine-tuned model improves mean SRCC from 0.651 to 0.722 and achieves the top gMAD rank, indicating strong adaptation and generalization.

Problem

• video quality assessment
• data selection
• model-data disconnect
• dataset curation
• active learning

Innovation

• model-informed data selection
• video quality assessment
• active fine-tuning
• failure prediction
• semantic diversity