Problem
Research questions and friction points this paper is trying to address.
neural video compression
B-frame coding
low-delay compression
state space models
entropy coding
DCVC-MB: Neural B-Frame Video Compression using State Space Models
🤖 AI Summary
This work addresses the low coding efficiency and high latency of B-frames in neural video compression by proposing a low-latency B-frame coding method based on an IBP frame structure. It introduces state space models into neural B-frame compression for the first time, designing a bidirectional spatiotemporal fusion mechanism to enable efficient temporal prediction. Additionally, an entropy-aware skipping mechanism is proposed to adaptively omit redundant latent representations. Combined with two inference optimization strategies, the approach significantly enhances compression performance. Experimental results demonstrate that the method achieves an average BD-rate reduction of 8.98% over existing neural codecs, outperforming VTM-19.0-LDP and VTM-19.0-RA by 30.45% and 1.81%, respectively.
📝 Abstract
In this paper we propose DCVC-Mamba (DCVC-MB), a neural video codec framework for B-frame coding. Our approach incorporates an IBP frame strategy for low-delay B-frame coding, a spatio-temporal fusion model based on state-space models for bidirectional temporal prediction, and an entropy-aware skipping mechanism that selectively omits coding certain latents to reduce entropy coding times. In addition to our model contributions we also implement two inference-time strategies that enhance compression performance. Experimental evaluation shows that DCVC-MB compares favorably to existing NVCs and traditional codecs. The method demonstrates BD-rate reductions of up to $8.98\%$ on average compared to prior neural video codecs, and improvements of up to $30.45\%$ and $1.81\%$ over the VTM-19.0-LDP and VTM-19.0-RA(Inter-GoP=16) benchmarks, respectively, contributing to advances in neural video compression.
Innovation
Methods, ideas, or system contributions that make the work stand out.
state-space models
neural video compression
B-frame coding
spatio-temporal fusion
entropy-aware skipping
🔎 Similar Papers
No similar papers found.
💼 Related Jobs
