Adaptive Inference-Time Scaling via Early-Step Latent Verification for Image Editing

2026-06-13

AI Summary

This work addresses the limitations of existing image editing methods, which often produce low-quality results in complex scenes due to the randomness of initial noise and struggle to balance efficiency and accuracy during inference. To overcome these challenges, we propose VeriLatent, a novel framework that introduces validity verification of initial noise using edit activation maps in the early latent space of diffusion models. VeriLatent further incorporates an adaptive inference budget allocation mechanism that dynamically adjusts computational resources according to editing difficulty. Through a plug-in design enabling efficient pruning, our method significantly reduces the number of function evaluations (NFE) while simultaneously improving both editing quality and inference speed across multiple benchmarks and base models, thereby breaking the longstanding trade-off between efficiency and accuracy in diffusion-based image editing.

Abstract

Instruction-based image editing has made notable progress with recent advances in generative models. However, the quality of the edited result is still influenced by the randomly sampled initial noise, particularly in complex editing scenarios. An unsuitable initial noise may lead to unsatisfactory editing results. Recent inference-time scaling methods address this issue by sampling multiple initial noises and selecting better candidates. Nevertheless, most of them follow a decode-then-verify scheme which introduces an efficiency-accuracy trade-off. When decoding is performed after limited inference steps, the decoded images often remain too noisy for reliable assessment, whereas sufficiently denoised images require much higher computational cost. To address this issue, we propose VeriLatent, a plug-and-play adaptive inference-time scaling framework with early-step latent verification for image editing. Specifically, we propose a novel verifier that scores each initial noise through a latent-space editing activation map at an early stage. It identifies promising candidates by assessing whether they can induce an effective edit in the correct region. This enables efficient early pruning without decoding latents into images. Building on this, we further develop an adaptive search strategy for inference-time scaling. It allocates inference budgets according to editing difficulty, thereby reducing the number of function evaluations (NFE). Extensive experiments on multiple benchmarks and different base models demonstrate that VeriLatent consistently improves both editing performance and inference-time scaling efficiency.

Problem

• image editing
• initial noise
• inference-time scaling
• efficiency-accuracy trade-off
• latent verification

Innovation

• inference-time scaling
• latent verification
• early-step pruning
• adaptive search
• image editing