Histogram-constrained Image Generation

2026-06-30

AI Summary

Existing diffusion models struggle to simultaneously ensure global semantic coherence and precise local control, particularly lacking effective constraints on the overall distribution of generated images—such as color or latent feature histograms. This work introduces, for the first time, distribution-level control into the diffusion generation process by leveraging optimal transport theory to apply explicit guidance transformations during sampling, thereby accurately aligning the output with a user-specified target histogram. The proposed method is compatible with existing control mechanisms and supports hybrid control strategies. It achieves high-quality image synthesis under constraints on both color and latent-space distributions, demonstrating its effectiveness and flexibility in tasks such as high-capacity information embedding.

Abstract

Diffusion models have emerged as a dominant paradigm in generative modeling, enabling high-fidelity sampling from complex data distributions. Despite impressive capabilities, controlling diffusion models to produce outputs aligned with user intent remains an open challenge, especially when balancing global coherence with local precision. Existing control mechanisms vary in the granularity of their conditioning signals. For example, textual prompts guide generation globally through high-level semantics, while ControlNet-like approaches secure precise local structure via dense conditions. In this work, we introduce Histogram-constrained Image Generation (HIG), a novel control mechanism that falls into the middle ground of control granularity. Our framework enforces user-specified distributional constraints (e.g., color histograms or latent token distributions) during the generation process with exact precision. We model such control as an optimal transport (OT) problem and apply explicit guidance transformations during sampling, thereby driving the diffusion trajectory to align with the desired histogram. We demonstrate the versatility of HIG across diverse applications, including constrained generation via color/latent histograms and high-capacity information embedding through histogram-level encoding. Our findings underscore the promise of distributional control, a flexible and interpretable control scheme that is fully compatible with existing control mechanisms, diversifying the hybrid strategies for controllable image generation. Our project page is available at: https://maps-research.github.io/hig/.

Problem

• diffusion models
• image generation
• distributional control
• histogram constraints
• controllable generation

Innovation

• Histogram-constrained Generation
• Optimal Transport
• Diffusion Models
• Distributional Control
• Controllable Image Synthesis