Higher fidelity perceptual image and video compression with a latent conditioned residual denoising diffusion model

2025-05-19

AI Summary

Existing diffusion-based image compression methods often sacrifice fidelity to improve perceptual quality. This paper proposes a hybrid compression framework that conditions on latent variables: first generating a high-fidelity initial reconstruction via a lightweight decoder, then enhancing perceptual quality using a residual denoising diffusion model. To our knowledge, this is the first work to introduce residual modeling into the diffusion compression paradigm, enabling joint optimization of PSNR and perceptual metrics (LPIPS/FID). The framework supports end-to-end training and naturally extends to video compression. On standard image benchmarks, it achieves up to 2.0 dB PSNR gain over CDC while maintaining comparable LPIPS and FID scores. Video compression experiments further demonstrate its generalizability and effectiveness across modalities.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Learning on the Edge & Model Compression
• Data Mining & Knowledge Management: Data Compression

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Denoising diffusion models achieved impressive results on several image generation tasks often outperforming GAN based models. Recently, the generative capabilities of diffusion models have been employed for perceptual image compression, such as in CDC. A major drawback of these diffusion-based methods is that, while producing impressive perceptual quality images they are dropping in fidelity/increasing the distortion to the original uncompressed images when compared with other traditional or learned image compression schemes aiming for fidelity. In this paper, we propose a hybrid compression scheme optimized for perceptual quality, extending the approach of the CDC model with a decoder network in order to reduce the impact on distortion metrics such as PSNR. After using the decoder network to generate an initial image, optimized for distortion, the latent conditioned diffusion model refines the reconstruction for perceptual quality by predicting the residual. On standard benchmarks, we achieve up to +2dB PSNR fidelity improvements while maintaining comparable LPIPS and FID perceptual scores when compared with CDC. Additionally, the approach is easily extensible to video compression, where we achieve similar results.

Problem

• Improving fidelity in perceptual image compression with diffusion models
• Reducing distortion while maintaining perceptual quality in compression
• Extending hybrid compression to video with similar performance gains

Innovation

• Hybrid compression with decoder network
• Latent conditioned residual diffusion model
• Improved PSNR while maintaining perceptual quality