FineVQ: Fine-Grained User Generated Content Video Quality Assessment

2024-12-26

AI Summary

To address the lack of fine-grained, interpretable analysis in user-generated content (UGC) video quality assessment, this paper proposes FineVQ—the first fine-grained UGC video quality assessment framework. It introduces FineVD, a large-scale open-source benchmark comprising 6,104 videos annotated with multi-dimensional quality scores and natural-language quality descriptions. We formally define and implement multi-dimensional interpretable quality attribution—e.g., blur, jitter, exposure—enabling root-cause analysis. FineVQ jointly models spatiotemporal video features and textual descriptions via dimension-wise disentanglement and differentiable quality decomposition, unifying rating prediction, scalar scoring, and attribution. Extensive experiments demonstrate state-of-the-art performance on FineVD and multiple mainstream UGC-VQA benchmarks, significantly improving both prediction accuracy and attribution plausibility. All code and data are publicly released.

Technology Category

• Computer Vision: Video Understanding & Activity Analysis
• Knowledge Representation and Reasoning: Qualitative Reasoning
• Machine Learning: Multimodal Learning

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

The rapid growth of user-generated content (UGC) videos has produced an urgent need for effective video quality assessment (VQA) algorithms to monitor video quality and guide optimization and recommendation procedures. However, current VQA models generally only give an overall rating for a UGC video, which lacks fine-grained labels for serving video processing and recommendation applications. To address the challenges and promote the development of UGC videos, we establish the first large-scale Fine-grained Video quality assessment Database, termed FineVD, which comprises 6104 UGC videos with fine-grained quality scores and descriptions across multiple dimensions. Based on this database, we propose a Fine-grained Video Quality assessment (FineVQ) model to learn the fine-grained quality of UGC videos, with the capabilities of quality rating, quality scoring, and quality attribution. Extensive experimental results demonstrate that our proposed FineVQ can produce fine-grained video-quality results and achieve state-of-the-art performance on FineVD and other commonly used UGC-VQA datasets. Both Both FineVD and FineVQ will be made publicly available.

Problem

• Video Quality Assessment
• User Generated Content
• Detail Quality Evaluation

Innovation

• FineVD Database
• FineVQ Model
• Detailed Video Quality Assessment