VoRTeC: Taming Foundation Flow for One-step Real time Video Compression

2026-09-02

AI Summary

为解决低比特率视频压缩中的模糊和延迟问题，提出VoRTeC框架，利用基础流模型实现一步解码，提高感知保真度与解码速度。

Abstract

Ultra-low bitrate video compression still faces critical challenges: traditional neural video compression inevitably introduces blurring artifacts, while diffusion-based generative video compression suffers from excessive decoding latency and poor temporal consistency. To address these issues, we propose $\mathtt{VoRTeC}$, a Video Compression framework built upon a foundational flow model (Wan2.1). By compactly encoding latent video representations, predicting the positions of compressed representations along flow trajectories, and integrating multi-scale priors, $\mathtt{VoRTeC}$ enables the compressor to harness generative video flow priors effectively. Without accessing the parameters or gradients of flow matching networks, our framework achieves one-step decoding and reconstructions with high perceptual fidelity. Meanwhile, we maintain consistency across frame groups via tail-frame reuse and prior caching. Extensive experiments demonstrate that our method reduces bit consumption by 58\% compared to prior diffusion-based approaches, with decoding speed boosted by 3 to 197 times: $\mathtt{VoRTeC}$ achieves a decoding speed of 13 FPS at 720p and 32 FPS at 480p.

Problem

• ultra-low bitrate
• video compression
• blurring artifacts
• decoding latency
• temporal consistency

Innovation

• foundational flow model
• one-step decoding
• perceptual fidelity
• temporal consistency
• bit consumption reduction