Model the Edit, Not the Image: Visual Autoregressive Editing from a Source-Centric Perspective

2026-08-09

AI Summary

Existing vision autoregressive (VAR) models for text-guided image editing typically adopt a generative-centric paradigm, which struggles to effectively leverage multi-scale representations of the source image and often relies on additional computation or manual intervention. This work proposes EditMod, the first method within the VAR framework to enable efficient, training-free, mask-free, and optimization-free editing. By comparing prediction discrepancies between source and target conditions within a shared autoregressive context, EditMod directly applies the resulting residual to source image tokens, focusing on localized, condition-driven changes rather than full-image reconstruction. The approach achieves strong textual alignment while preserving high source-image fidelity, enabling end-to-end editing of a single 1K-resolution image in just 1.57 seconds on an A100 GPU without per-image preprocessing.

Abstract

Next-scale visual autoregressive models (VARs) have emerged as a powerful generative paradigm, producing high-quality images through efficient coarse-to-fine prediction. However, their potential for text-guided image editing remains largely underexplored. Existing training-free VAR editing approaches often formulate editing as target-conditioned regeneration guided or constrained by the source image, and may rely on inversion, test-time optimization, attention control, or user-provided masks. This generation-centric formulation does not fully exploit the multiscale source representations provided by VARs and may introduce additional computation or intervention. We instead take a source-centric perspective on VAR editing, in which the encoded source image tokens serve as the primary visual state and the editing process focuses on condition-induced changes. Based on this perspective, we propose \textbf{EditMod}, which compares source- and target-conditioned predictions under a shared autoregressive context, treats their difference as a scale-wise editing direction, and applies it as a residual update to source tokens at selected scales. Experiments show that EditMod achieves leading source-image fidelity while maintaining strong text alignment, and completes end-to-end editing of a 1K image in only 1.57 seconds on a single A100 GPU without per-image preparation.

Problem

• visual autoregressive models
• image editing
• source-centric editing
• text-guided editing
• multiscale representation

Innovation

• visual autoregressive editing
• source-centric editing
• EditMod
• scale-wise residual update
• text-guided image editing