BiECVC: Gated Diversification of Bidirectional Contexts for Learned Video Compression

2025-05-14

AI Summary

Existing learning-based bidirectional video compression (BVC) lags behind forward-only approaches primarily due to insufficient contextual modeling and difficulty in adaptively suppressing harmful information caused by fast motion and occlusion. This paper proposes BiECVC, the first BVC framework featuring a bidirectional context gating mechanism that jointly integrates local feature reuse and linear-attention-driven non-local modeling. It employs decoded motion vectors for zero-overhead feature alignment and introduces a data-dependent decay strategy to enhance gating robustness. Experimental results under random-access (RA) configuration demonstrate that BiECVC achieves BD-rate reductions of 13.4% and 15.7% over VTM 13.2 (with IDR intervals of 32 and 64, respectively), marking the first learned video compression (LVC) method to consistently outperform VTM 13.2 across all standard test sequences.

Technology Category

• Computer Vision: Video Understanding & Activity Analysis
• Machine Learning: Learning on the Edge & Model Compression
• Data Mining & Knowledge Management: Data Compression

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Recent forward prediction-based learned video compression (LVC) methods have achieved impressive results, even surpassing VVC reference software VTM under the Low Delay B (LDB) configuration. In contrast, learned bidirectional video compression (BVC) remains underexplored and still lags behind its forward-only counterparts. This performance gap is mainly due to the limited ability to extract diverse and accurate contexts: most existing BVCs primarily exploit temporal motion while neglecting non-local correlations across frames. Moreover, they lack the adaptability to dynamically suppress harmful contexts arising from fast motion or occlusion. To tackle these challenges, we propose BiECVC, a BVC framework that incorporates diversified local and non-local context modeling along with adaptive context gating. For local context enhancement, BiECVC reuses high-quality features from lower layers and aligns them using decoded motion vectors without introducing extra motion overhead.To model non-local dependencies efficiently, we adopt a linear attention mechanism that balances performance and complexity. To further mitigate the impact of inaccurate context prediction, we introduce Bidirectional Context Gating, inspired by data-dependent decay in recent autoregressive language models, to dynamically filter contextual information based on conditional coding results. Extensive experiments demonstrate that BiECVC achieves state-of-the-art performance, reducing the bit-rate by 13.4% and 15.7% compared to VTM 13.2 under the Random Access (RA) configuration with intra periods of 32 and 64, respectively. To our knowledge, BiECVC is the first learned video codec to surpass VTM 13.2 RA across all standard test datasets. Code will be available at https://github.com/JiangWeibeta/ECVC.

Problem

• Enhancing bidirectional video compression with diverse context modeling
• Addressing limited local and non-local correlation extraction in BVC
• Dynamically filtering harmful contexts in fast motion or occlusion

Innovation

• Diversified local and non-local context modeling
• Linear attention mechanism for efficiency
• Bidirectional Context Gating for dynamic filtering