Bit Allocation Transfer for Perceptual Quality Enhancement of Traditional Video Codecs

2026-08-07

AI Summary

Traditional video encoders employing mean squared error (MSE)-based rate-distortion optimization struggle to enhance perceptual quality as perceived by the human visual system. This work proposes a bit allocation transfer framework that, for the first time, implicitly transfers perceptual importance learned by a neural video compression model (based on DCVC-FM) to conventional encoders. Specifically, block-level quantization step maps are derived from motion-compensated predicted frames and converted into QP maps to guide block-level bit allocation in H.264/AVC, H.265/HEVC, and H.266/VVC, without requiring any modifications to the decoder syntax. Evaluated on the HEVC Class B dataset, the proposed method achieves MS-SSIM-based bitrate savings of 20.20%, 8.25%, and 8.37% over JM-19.0, HM-16.20, and VTM-23.0, respectively, with further gains observed when incorporating the predicted frame into the optimization process.

Abstract

Traditional block-based video codecs, such as H.264/AVC, H.265/HEVC and H.266/VVC, rely on hand-crafted Rate-Distortion Optimization (RDO) processes that primarily minimize Mean Squared Error (MSE), which correlates poorly with human perceptual quality. While neural video compression methods can easily optimize perceptually aligned metrics like MS-SSIM, their high computational complexity limits practical deployment. This paper proposes a novel bit allocation transfer framework that bridges these two paradigms to enhance the perceptual quality of conventional video codecs. Specifically, we train a quantization step generation model using a perceptual loss within a neural video compression framework (DCVC-FM). The model takes the original frame and a motion-compensated prediction as input and outputs a quantization step map. We then derive a block-wise bit ratio from this map and convert it into a Quantization Parameter (QP) map for a traditional video codec. Experimental results on the HEVC B$\sim$D dataset demonstrate that our method achieves 20.20\%, 8.25\%, and 8.37\% bitrate savings in terms of MS-SSIM compared with the standard reference software JM-19.0, HM-16.20, and VTM-23.0, respectively, with additional gains when utilizing predicted frames. Our approach effectively transfers the implicit perceptual importance learned by neural video compression models to guide block-level bit allocation in traditional video codecs without modifying their core decoding syntax.

Problem

• perceptual quality
• video codecs
• bit allocation
• rate-distortion optimization
• MSE

Innovation

• bit allocation transfer
• perceptual quality enhancement
• neural video compression
• quantization parameter map
• traditional video codecs