Compressed Image Generation with Denoising Diffusion Codebook Models

2025-02-03

AI Summary

This paper addresses the challenge of jointly modeling image generation and compression. We propose the Denoising Diffusion Codebook Model (DDCM), which replaces continuous noise sampling in the reverse diffusion process with a predefined i.i.d. Gaussian codebook, enabling simultaneous high-fidelity image generation and lossless bitstream output. Our key contribution is the first deep integration of diffusion models with compressible discrete codebooks, underpinned by a noise-matching selection mechanism and score-based posterior sampling theory to ensure reconstruction fidelity. DDCM achieves FID and LPIPS scores comparable to standard denoising diffusion models (DDMs) using only a minimal codebook size, and natively supports entropy coding. By unifying generative and compressive modeling within a single framework, DDCM seamlessly extends to conditional generation tasks such as image inpainting. On ImageNet, it establishes new state-of-the-art performance in perceptual compression.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Data Mining & Knowledge Management: Data Compression

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

We present a novel generative approach based on Denoising Diffusion Models (DDMs), which produces high-quality image samples along with their losslessly compressed bit-stream representations. This is obtained by replacing the standard Gaussian noise sampling in the reverse diffusion with a selection of noise samples from pre-defined codebooks of fixed iid Gaussian vectors. Surprisingly, we find that our method, termed Denoising Diffusion Codebook Model (DDCM), retains sample quality and diversity of standard DDMs, even for extremely small codebooks. We leverage DDCM and pick the noises from the codebooks that best match a given image, converting our generative model into a highly effective lossy image codec achieving state-of-the-art perceptual image compression results. More generally, by setting other noise selections rules, we extend our compression method to any conditional image generation task (e.g., image restoration), where the generated images are produced jointly with their condensed bit-stream representations. Our work is accompanied by a mathematical interpretation of the proposed compressed conditional generation schemes, establishing a connection with score-based approximations of posterior samplers for the tasks considered.

Problem

• Image Compression
• Quality Preservation
• Detail Retention

Innovation

• DDCM
• Image Compression
• Noise Optimization