Coding-Prior Guided Diffusion Network for Video Deblurring

2025-04-16

AI Summary

Existing video deblurring methods largely ignore motion vectors (MVs) and coding residuals (CRs) embedded in video codecs, as well as the realistic priors encoded in pre-trained diffusion models. To address this, we propose a two-stage encoding-diffusion collaborative framework. In the first stage, MVs and CRs are leveraged to generate optical-flow-aligned features and residual-driven attention masks. In the second stage, encoding priors are injected into a diffusion model via CPFP (Cross-Phase Feature Propagation) for temporal feature alignment and CPC (Controlled Prior Conditioning) for region-aware enhancement and fine-grained detail reconstruction. This work is the first to jointly exploit video encoding priors and diffusion-based generative priors, effectively mitigating motion estimation errors and texture distortion. Our method achieves up to a 30% improvement in IQA metrics, setting new state-of-the-art perceptual quality. The source code and a novel encoding-prior-enhanced benchmark dataset are publicly released.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

While recent video deblurring methods have advanced significantly, they often overlook two valuable prior information: (1) motion vectors (MVs) and coding residuals (CRs) from video codecs, which provide efficient inter-frame alignment cues, and (2) the rich real-world knowledge embedded in pre-trained diffusion generative models. We present CPGDNet, a novel two-stage framework that effectively leverages both coding priors and generative diffusion priors for high-quality deblurring. First, our coding-prior feature propagation (CPFP) module utilizes MVs for efficient frame alignment and CRs to generate attention masks, addressing motion inaccuracies and texture variations. Second, a coding-prior controlled generation (CPC) module network integrates coding priors into a pretrained diffusion model, guiding it to enhance critical regions and synthesize realistic details. Experiments demonstrate our method achieves state-of-the-art perceptual quality with up to 30% improvement in IQA metrics. Both the code and the codingprior-augmented dataset will be open-sourced.

Problem

• Leveraging motion vectors and coding residuals for video deblurring
• Integrating coding priors with diffusion models for realistic details
• Improving perceptual quality in video deblurring by 30%

Innovation

• Utilizes motion vectors for frame alignment
• Integrates coding residuals for attention masks
• Guides diffusion model with coding priors