Real-Time Neural Video Compression with Unified Intra and Inter Coding

2025-10-16

AI Summary

Existing neural video compression (NVC) methods suffer from limited performance in handling occlusion disocclusion, introducing novel content, and mitigating inter-frame error propagation. To address these challenges, this paper proposes a unified intra/inter coding framework that enables adaptive mode switching within a single model, along with a bidirectional two-frame joint compression mechanism that jointly exploits forward and backward temporal redundancies to significantly suppress error accumulation. Leveraging an end-to-end deep architecture and joint optimization strategy, the method enhances compression stability and visual consistency while maintaining real-time efficiency. Experimental results demonstrate that our approach achieves an average 10.7% BD-rate reduction over DCVC-RT, with notably reduced frame-level bitrate and quality fluctuations. The proposed method thus delivers efficient, robust, and adaptive real-time video compression.

Technology Category

• Computer Vision: Video Understanding & Activity Analysis
• Machine Learning: Learning on the Edge & Model Compression
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Neural video compression (NVC) technologies have advanced rapidly in recent years, yielding state-of-the-art schemes such as DCVC-RT that offer superior compression efficiency to H.266/VVC and real-time encoding/decoding capabilities. Nonetheless, existing NVC schemes have several limitations, including inefficiency in dealing with disocclusion and new content, interframe error propagation and accumulation, among others. To eliminate these limitations, we borrow the idea from classic video coding schemes, which allow intra coding within inter-coded frames. With the intra coding tool enabled, disocclusion and new content are properly handled, and interframe error propagation is naturally intercepted without the need for manual refresh mechanisms. We present an NVC framework with unified intra and inter coding, where every frame is processed by a single model that is trained to perform intra/inter coding adaptively. Moreover, we propose a simultaneous two-frame compression design to exploit interframe redundancy not only forwardly but also backwardly. Experimental results show that our scheme outperforms DCVC-RT by an average of 10.7% BD-rate reduction, delivers more stable bitrate and quality per frame, and retains real-time encoding/decoding performances. Code and models will be released.

Problem

• Addresses disocclusion and new content inefficiency in neural video compression
• Eliminates interframe error propagation without manual refresh mechanisms
• Enhances compression efficiency while maintaining real-time encoding and decoding

Innovation

• Unified intra and inter coding in neural video compression
• Simultaneous two-frame compression for bidirectional redundancy
• Single model adaptively handles intra and inter coding