Multi-Conditioned Diffusion Synthesis of Sand Boils for Low-Resource Earthen-Levee Inspection

2026-07-07

AI Summary

This work addresses the scarcity of real annotated data for detecting sand boil defects in earthen levees by proposing a diffusion model–based synthetic data augmentation approach. Leveraging Stable Diffusion XL fine-tuned with DreamBooth, the method integrates multi-branch ControlNet, Prompt Atlas, and soft-mask inpainting to repaint backgrounds while preserving authentic defect structures, thereby avoiding stitching artifacts and enabling cross-category transfer. Requiring only a limited number of real samples, the framework generates high-quality, label-traceable synthetic images. From an initial set of 1,020 candidates, 815 high-fidelity samples were selected, achieving a strong balance among fidelity, diversity, and label reliability. The study also releases preset configurations and a hybrid augmented dataset to support future research.

Abstract

Sand boils on earthen levees are safety-critical defects, but pixel-level detection is limited by scarce annotations. We present a diffusion-based synthesis pipeline for low-resource sand-boil imagery. Using Stable Diffusion XL fine-tuned with DreamBooth and conditioned by a multi-branch ControlNet stack, the pipeline generates synthetic inspection images from a small curated reference set. A soft-mask inpainting protocol preserves the real defect pixels while re-rendering the surrounding scene, avoiding seams and color shifts from prior seamless-cloning compositing. A mask-conditioned ControlNet can also generate a new boil inside a chosen mask, making the mask the segmentation label by construction; however, because large-scale label certification remains unresolved with the available real-trained gate, we release the soft-mask preset as the default. Text conditioning is supplied by a taxonomy-driven Prompt Atlas that expands one domain specification into a stratified, CLIP-validated prompt bank and transfers to new defect classes without code changes. From the real training images, the pipeline produces 1,020 synthetic candidates, of which 815 pass a CLIP admissibility filter. We evaluate image quality using distributional and fidelity-diversity measures against the real reference set and a Poisson baseline, and audit for out-of-distribution drift and memorization. No single preset dominates; each trades off fidelity, diversity, and label reliability. We therefore release the label-reliable preset as the default and treat a curated mixture as the natural augmentation set. Our claims are limited to image quality, label provenance, and diversity; downstream segmentation is left for future work. Code and an artifact manifest are released for reproducibility.

Problem

• sand boils
• earthen levees
• low-resource
• pixel-level detection
• scarce annotations

Innovation

• diffusion synthesis
• soft-mask inpainting
• multi-conditioned ControlNet
• Prompt Atlas
• low-resource image generation