🤖 AI Summary
To address the challenge of no-reference, multi-generation compressed video quality assessment in user-generated content (UGC), this paper proposes a novel no-reference video quality assessment (NR-VQA) method. The approach innovatively jointly models residual frame patches and optical flow, integrating multi-level feature stacking from both CNN (ResNet) and Vision Transformer (ViT) architectures to jointly enhance spatiotemporal perception and semantic abstraction. Key components include residual frame extraction, optical flow estimation, multi-scale spatial representation learning, and cross-architecture hierarchical feature fusion, all optimized end-to-end. Evaluated on four mainstream UGC datasets, the method achieves average Spearman rank-order correlation coefficient (SRCC) of 0.8658 and Pearson linear correlation coefficient (PLCC) of 0.8872—substantially outperforming state-of-the-art methods. The source code and pre-trained models are publicly available.
📝 Abstract
With the rapid growth of User-Generated Content (UGC) exchanged between users and sharing platforms, the need for video quality assessment in the wild has emerged. UGC is mostly acquired using consumer devices and undergoes multiple rounds of compression or transcoding before reaching the end user. Therefore, traditional quality metrics that require the original content as a reference cannot be used. In this paper, we propose ReLaX-VQA, a novel No-Reference Video Quality Assessment (NR-VQA) model that aims to address the challenges of evaluating the diversity of video content and the assessment of its quality without reference videos. ReLaX-VQA uses fragments of residual frames and optical flow, along with different expressions of spatial features of the sampled frames, to enhance motion and spatial perception. Furthermore, the model enhances abstraction by employing layer-stacking techniques in deep neural network features (from Residual Networks and Vision Transformers). Extensive testing on four UGC datasets confirms that ReLaX-VQA outperforms existing NR-VQA methods with an average SRCC value of 0.8658 and PLCC value of 0.8872. We will open source the code and trained models to facilitate further research and applications of NR-VQA: https://github.com/xinyiW915/ReLaX-VQA.