Tuning-Free Latent Diffusion Models for Ultrahigh-Resolution Image Editing

2026-07-07

AI Summary

Existing diffusion models struggle to directly edit ultra-high-resolution images such as 8K due to memory constraints and the high cost of training on high-resolution data, while naively upscaling low-resolution outputs often results in blurriness and loss of detail. This work proposes UltraDiffEdit, a framework that enables off-the-shelf latent diffusion models (LDMs) to perform high-quality 8K image editing without any fine-tuning. By integrating multi-scale progressive editing, multi-patch latent encoding, global-local consistency-aware denoising, and a patch-based hybrid sampling strategy, UltraDiffEdit achieves, for the first time, training-free 8K image editing on a single consumer-grade GPU (e.g., RTX 3090), significantly outperforming existing methods in detail preservation, boundary smoothness, and semantic consistency.

Abstract

Recent diffusion-based generative models have shown impressive performance in image generation and editing. However, due to memory limitations and the high cost of collecting high-resolution training images, existing methods are typically restricted to inputs with linear resolutions below 1K. In contrast, photos captured by modern mobile devices often reach linear resolutions up to 8K, revealing a significant gap between current capabilities and real-world demands. Simply upscaling low-resolution edited results often results in visually enlarged but blurry images that lack fine details. This paper introduces UltraDiffEdit, a novel, tuning-free image editing framework that extends off-the-shelf latent diffusion models (LDMs) to ultrahigh resolutions. UltraDiffEdit employs a multi-scale progressive editing strategy, iteratively blending high-resolution edited content with unedited areas in a coarse-to-fine manner. We employ multi-patch encoding to preserve both edited and unedited visual details within the latent space. To mitigate editing artifacts, our global-local consistency denoising technique consistently integrates edited and unedited latent features, ensuring smooth transition at editing boundaries from the latent representation to the final image. We also introduce a patch-based hybrid sampling approach that captures local, intermediate, and global features, ensuring semantic coherence and enhancing fine detail during denoising. We conduct extensive experiments demonstrating UltraDiffEdit's superior editing quality and flexibility: it can handle image resolutions up to 8K using only a single NVIDIA GeForce RTX 3090 GPU. The source code is publicly available at https://github.com/LonglongaaaGo/UltraDiffEdit.

Problem

• ultrahigh-resolution image editing
• latent diffusion models
• memory limitation
• high-resolution generation
• image detail preservation

Innovation

• tuning-free
• ultrahigh-resolution
• latent diffusion models
• multi-scale editing
• patch-based sampling