PixPerfect: Seamless Latent Diffusion Local Editing with Discriminative Pixel-Space Refinement

2025-12-02

AI Summary

Latent diffusion models (LDMs) suffer from pixel-level inconsistencies—including color shifts, texture mismatches, and boundary seams—in local image editing due to aggressive latent-space compression. Existing approaches struggle to balance generality and fidelity. To address this, we propose a universal, pixel-level refinement framework comprising three key components: (1) a differentiable discriminative pixel-space modeling module that explicitly enforces fine-grained detail consistency; (2) a background-aware latent decoding mechanism coupled with adversarial artifact simulation during training to suppress structural artifacts; and (3) a plug-and-play direct pixel-space optimization module compatible with diverse implicit representations and editing tasks. Extensive experiments on image inpainting, object removal, and object insertion demonstrate substantial improvements in visual fidelity. Our method achieves state-of-the-art performance across multiple benchmarks while exhibiting strong generalization and practical applicability.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Latent Diffusion Models (LDMs) have markedly advanced the quality of image inpainting and local editing. However, the inherent latent compression often introduces pixel-level inconsistencies, such as chromatic shifts, texture mismatches, and visible seams along editing boundaries. Existing remedies, including background-conditioned latent decoding and pixel-space harmonization, usually fail to fully eliminate these artifacts in practice and do not generalize well across different latent representations or tasks. We introduce PixPerfect, a pixel-level refinement framework that delivers seamless, high-fidelity local edits across diverse LDM architectures and tasks. PixPerfect leverages (i) a differentiable discriminative pixel space that amplifies and suppresses subtle color and texture discrepancies, (ii) a comprehensive artifact simulation pipeline that exposes the refiner to realistic local editing artifacts during training, and (iii) a direct pixel-space refinement scheme that ensures broad applicability across diverse latent representations and tasks. Extensive experiments on inpainting, object removal, and insertion benchmarks demonstrate that PixPerfect substantially enhances perceptual fidelity and downstream editing performance, establishing a new standard for robust and high-fidelity localized image editing.

Problem

• Addresses pixel-level inconsistencies in latent diffusion image editing
• Eliminates chromatic shifts, texture mismatches, and visible seams
• Ensures seamless local edits across diverse LDM architectures and tasks

Innovation

• Differentiable discriminative pixel space amplifies and suppresses discrepancies
• Comprehensive artifact simulation pipeline exposes refiner to realistic artifacts
• Direct pixel-space refinement ensures broad applicability across tasks