StippleDiffusion: Capacity-Constrained Stippling using Controlled Diffusion

2026-05-15

AI Summary

This work proposes the first diffusion-based framework for conditional stippling generation, addressing limitations of existing methods that either rely on time-consuming per-image optimization or struggle with capacity constraints under image guidance. By integrating optimal transport-based point-set diffusion with a ControlNet architecture, the approach enables differentiable generation from arbitrary-density inputs and generalizes to unseen point counts. Key innovations include density-weighted rejection sampling for initialization, a Sigmoid-gated 1×1 convolution injection mechanism, and explicit incorporation of capacity constraints during late-stage denoising to preserve blue-noise structure. Evaluated on the Icons-50 benchmark, the method matches the quality of per-density optimization approaches while achieving near-constant inference time regardless of output point count and supporting end-to-end training.

Abstract

Stipple patterns, point sets whose local density tracks a target image, are traditionally produced by per-density iterative optimizers, which are slow, non-differentiable, and must be re-run from scratch for each new target. Learned alternatives have so far addressed only unconditional point generation; capacity-constrained, image-conditioned stippling has remained out of reach. We present the first diffusion-based sampler that simultaneously satisfies a learned local point-distribution prior and a continuous, image-defined capacity constraint at inference. The method is a ControlNet branch built on top of an optimal-transport-grid point-set diffusion baseline, conditioned on the target density map and a high-resolution image. Two design choices make the combination tractable: training and inference are restricted to the late-stage denoising regime, initialized from a density-weighted rejection sample, and the standard zero-convolution injection is replaced with a sigmoid-gated 1x1 projection that preserves the base model's blue-noise structure under hard density signals. A single trained checkpoint accepts arbitrary target densities at inference, generalizes to point budgets that were not seen during training, and produces stipples in time nearly independent of the output point count. On the Icons-50 benchmark, our learned sampler reaches parity with per-density-optimized baselines on every reported metric while remaining differentiable end-to-end.

Problem

• stippling
• capacity-constrained
• image-conditioned
• point set generation
• density control

Innovation

• diffusion model
• capacity-constrained stippling
• ControlNet
• optimal transport
• blue-noise sampling