GuidPaint: Class-Guided Image Inpainting with Diffusion Models

2025-07-29

AI Summary

Existing context-aware diffusion inpainting methods require no training but lack fine-grained semantic control over masked regions, often yielding semantically implausible content; conversely, architecture-modification-based approaches incur prohibitive retraining costs. To address this, we propose GuidPaint—a training-free, class-guided image inpainting framework. GuidPaint is the first to integrate classifier guidance into context-aware diffusion inpainting, enabling precise semantic control over generated content in masked regions during the denoising process. It further introduces a hybrid sampling strategy combining stochastic and deterministic components, supporting interactive user selection and iterative refinement of intermediate results. Experiments demonstrate that, while maintaining zero-shot capability, GuidPaint significantly improves semantic consistency and visual fidelity of inpainted results. Both qualitative assessment and quantitative metrics (e.g., LPIPS, FID, user study scores) surpass those of existing training-free methods.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

In recent years, diffusion models have been widely adopted for image inpainting tasks due to their powerful generative capabilities, achieving impressive results. Existing multimodal inpainting methods based on diffusion models often require architectural modifications and retraining, resulting in high computational cost. In contrast, context-aware diffusion inpainting methods leverage the model's inherent priors to adjust intermediate denoising steps, enabling high-quality inpainting without additional training and significantly reducing computation. However, these methods lack fine-grained control over the masked regions, often leading to semantically inconsistent or visually implausible content. To address this issue, we propose GuidPaint, a training-free, class-guided image inpainting framework. By incorporating classifier guidance into the denoising process, GuidPaint enables precise control over intermediate generations within the masked areas, ensuring both semantic consistency and visual realism. Furthermore, it integrates stochastic and deterministic sampling, allowing users to select preferred intermediate results and deterministically refine them. Experimental results demonstrate that GuidPaint achieves clear improvements over existing context-aware inpainting methods in both qualitative and quantitative evaluations.

Problem

• Lack of fine-grained control in diffusion-based inpainting
• Semantic inconsistency in masked region generation
• High computational cost of retraining multimodal models

Innovation

• Training-free class-guided inpainting framework
• Classifier-guided denoising for semantic control
• Hybrid stochastic-deterministic sampling refinement