A Rate-Quality Model for Learned Video Coding

2025-05-05

AI Summary

This paper addresses the inaccurate modeling and poor adaptability of rate–quality (R–Q) relationships to dynamic content variations in learned video coding (LVC). To this end, we propose RQNet—the first fully online adaptive, parameterized R–Q model for LVC. RQNet employs a neural network to dynamically predict the R–Q mapping and incorporates historical frame features for context-aware real-time rate control. It further introduces a least-squares-based online parameter estimation and dynamic update mechanism, enabling continuous model refinement during encoding. Evaluated on mainstream datasets, RQNet significantly reduces bitrate deviation (average reduction of 38.2%) and substantially improves R–Q prediction accuracy, while incurring less than 0.5% additional computational overhead. Our core contribution lies in establishing the first content-aware, lightweight, and fully online adaptive R–Q modeling framework for LVC—bridging a critical gap between theoretical R–Q characterization and practical, dynamic coding scenarios.

Technology Category

• Computer Vision: Learning & Optimization for CV
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Machine Learning: Learning on the Edge & Model Compression

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: User modeling for targeted and personalized online advertising
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Learned video coding (LVC) has recently achieved superior coding performance. In this paper, we model the rate-quality (R-Q) relationship for learned video coding by a parametric function. We learn a neural network, termed RQNet, to characterize the relationship between the bitrate and quality level according to video content and coding context. The predicted (R,Q) results are further integrated with those from previously coded frames using the least-squares method to determine the parameters of our R-Q model on-the-fly. Compared to the conventional approaches, our method accurately estimates the R-Q relationship, enabling the online adaptation of model parameters to enhance both flexibility and precision. Experimental results show that our R-Q model achieves significantly smaller bitrate deviations than the baseline method on commonly used datasets with minimal additional complexity.

Problem

• Modeling rate-quality relationship in learned video coding
• Predicting bitrate and quality via neural network RQNet
• Enhancing online parameter adaptation for better precision

Innovation

• Parametric function models rate-quality relationship
• RQNet neural network predicts bitrate and quality
• Least-squares method integrates frame coding results