TINQ: Temporal Inconsistency Guided Blind Video Quality Assessment

2024-12-25

AI Summary

To address temporal inconsistency—the dominant quality degradation factor in user-generated content (UGC) and super-resolution (SR) videos—this paper proposes the first no-reference video quality assessment (NR-VQA) method tailored specifically for UGC/SR scenarios. The method comprises: (i) a spatial multi-granularity anomaly detection module to localize inter-frame inconsistent regions; and (ii) a two-stage temporal module that models multi-scale dynamic features, incorporating a vision-memory-capacity-driven temporal segmentation strategy and a consistency-aware fusion unit for cross-scale quality perception and adaptive aggregation. Its key innovation lies in the first systematic differentiation and modeling of distinct temporal distortion patterns inherent to UGC versus SR videos. Extensive evaluations on mainstream UGC and SR benchmark datasets demonstrate consistent and significant improvements over state-of-the-art methods, with substantial gains in PLCC and SRCC. The source code is publicly available.

Technology Category

• Computer Vision: Video Understanding & Activity Analysis
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Web data quality in the era of algorithmically-generated content
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs

Abstract

Blind video quality assessment (BVQA) has been actively researched for user-generated content (UGC) videos. Recently, super-resolution (SR) techniques have been widely applied in UGC. Therefore, an effective BVQA method for both UGC and SR scenarios is essential. Temporal inconsistency, referring to irregularities between consecutive frames, is relevant to video quality. Current BVQA approaches typically model temporal relationships in UGC videos using statistics of motion information, but inconsistencies remain unexplored. Additionally, different from temporal inconsistency in UGC videos, such inconsistency in SR videos is amplified due to upscaling algorithms. In this paper, we introduce the Temporal Inconsistency Guided Blind Video Quality Assessment (TINQ) metric, demonstrating that exploring temporal inconsistency is crucial for effective BVQA. Since temporal inconsistencies vary between UGC and SR videos, they are calculated in different ways. Based on this, a spatial module highlights inconsistent areas across consecutive frames at coarse and fine granularities. In addition, a temporal module aggregates features over time in two stages. The first stage employs a visual memory capacity block to adaptively segment the time dimension based on estimated complexity, while the second stage focuses on selecting key features. The stages work together through Consistency-aware Fusion Units to regress cross-time-scale video quality. Extensive experiments on UGC and SR video quality datasets show that our method outperforms existing state-of-the-art BVQA methods. Code is available at https://github.com/Lighting-YXLI/TINQ.

Problem

• Video Quality Assessment
• Super-Resolution Techniques
• Temporal Inconsistency

Innovation

• Blind Video Quality Assessment
• Temporal Inconsistency
• Super-Resolution Artifact Detection