Towards Robust Sequential Decomposition for Complex Image Editing

2026-05-09

AI Summary

Existing image editing models struggle with complex instructions involving compositional operations or step dependencies: single-step editing fails to accurately parse such instructions, while sequential editing suffers from error accumulation and resulting distortions. This work proposes a unified contextual editing framework that leverages a synthetic data pipeline to generate multi-level complex editing tasks and employs high-quality decomposition sequences for instruction fine-tuning. Innovatively adopting a synergistic training strategy combining synthetic and real data, the study demonstrates for the first time that decomposition capabilities learned from synthetic tasks effectively transfer to real images, achieving sim-to-real generalization in complex editing scenarios. Experiments show that the method maintains robust performance as task complexity increases, significantly enhancing the model’s ability to understand intricate instructions and execute them with high fidelity.

Abstract

Recent advances in visual generative models have enabled high-fidelity image editing guided by human instructions. However, these models often struggle with complex instructions involving combinatorial editing operations or inter-step dependencies. This difficulty stems from the limitations of two canonical paradigms: (1) single-turn editing, which attempts to apply all instructed edits in one pass, often fails to parse the complex instruction accurately and causes undesired edits; and (2) sequential editing can decompose the task into simpler steps but suffers from compounding errors introduced by the sequential execution, leading to low-fidelity results. To derive a robust solution for complex image editing, we examine editing behaviors of different paradigms under a unified in-context editing framework, and study how the benefits of sequential decomposition can be balanced against its error-accumulation drawbacks. We further develop a synthetic data pipeline that constructs editing tasks of varying instruction complexity, allowing us to curate a large-scale editing dataset with high-quality decomposed sequences. By finetuning on synthetic data, we discovered that with properly designed editing paradigms, sequential decomposition yields robust improvements even as task complexity increases. Furthermore, the decomposition skills learned from synthetic tasks can transfer to real images by co-training with real-world editing data, demonstrating the promise of sim-to-real generalization for tackling complex image editing across broader domains.

Problem

• complex image editing
• sequential decomposition
• error accumulation
• instruction complexity
• visual generative models

Innovation

• sequential decomposition
• in-context editing
• synthetic data pipeline
• sim-to-real generalization
• complex image editing