Binary Diffusion Probabilistic Model

2025-01-23

AI Summary

Existing continuous-assumption-based denoising diffusion probabilistic models (DDPMs) struggle to adapt to binary data (e.g., pixel bits) and suffer from inefficient training for generation and inpainting tasks. To address this, we propose the first diffusion model explicitly designed for binary data: it decomposes images into bit planes, replaces Gaussian noise injection with an XOR-based noise transformation, and employs binary cross-entropy loss for end-to-end optimization. This formulation circumvents continuous modeling constraints, enabling precise discrete noise control, faster convergence, and significantly reduced sampling steps. Evaluated on FFHQ, CelebA, and CelebA-HQ, our method achieves state-of-the-art performance across super-resolution, image inpainting, and blind restoration—outperforming prior approaches in both fidelity and efficiency. Notably, it reduces inference steps by over 50% and substantially lowers computational overhead, demonstrating the effectiveness and practicality of binary-diffusion modeling.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

We introduce the Binary Diffusion Probabilistic Model (BDPM), a novel generative model optimized for binary data representations. While denoising diffusion probabilistic models (DDPMs) have demonstrated notable success in tasks like image synthesis and restoration, traditional DDPMs rely on continuous data representations and mean squared error (MSE) loss for training, applying Gaussian noise models that may not be optimal for discrete or binary data structures. BDPM addresses this by decomposing images into bitplanes and employing XOR-based noise transformations, with a denoising model trained using binary cross-entropy loss. This approach enables precise noise control and computationally efficient inference, significantly lowering computational costs and improving model convergence. When evaluated on image restoration tasks such as image super-resolution, inpainting, and blind image restoration, BDPM outperforms state-of-the-art methods on the FFHQ, CelebA, and CelebA-HQ datasets. Notably, BDPM requires fewer inference steps than traditional DDPM models to reach optimal results, showcasing enhanced inference efficiency.

Problem

• Binary Data
• Diffusion Models
• Training Optimization

Innovation

• Binary Diffusion Probability Model
• Binary Data Handling
• Enhanced Image Restoration