MCIE: Multimodal LLM-Driven Complex Instruction Image Editing with Spatial Guidance

2026-02-08

AI Summary

This work addresses the limitations of existing image editing methods in accurately following complex, compositional natural language instructions while preserving background consistency. To this end, we propose MCIE-E1, a novel approach that leverages spatial-aware and background-consistent cross-attention mechanisms during the denoising process to achieve precise spatial alignment between semantic instructions and corresponding image regions, while retaining features in unedited areas to maintain visual coherence. Our key contributions include a spatial-guidance module that enhances comprehension of intricate instructions, a background consistency module ensuring global visual harmony, and CIE-Bench—the first benchmark specifically designed for evaluating complex instruction-based image editing. Experimental results demonstrate that MCIE-E1 achieves a 23.96% improvement in instruction-following accuracy on CIE-Bench, significantly outperforming current state-of-the-art methods in both quantitative and qualitative evaluations.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

Recent advances in instruction-based image editing have shown remarkable progress. However, existing methods remain limited to relatively simple editing operations, hindering real-world applications that require complex and compositional instructions. In this work, we address these limitations from the perspectives of architectural design, data, and evaluation protocols. Specifically, we identify two key challenges in current models: insufficient instruction compliance and background inconsistency. To this end, we propose MCIE-E1, a Multimodal Large Language Model-Driven Complex Instruction Image Editing method that integrates two key modules: a spatial-aware cross-attention module and a background-consistent cross-attention module. The former enhances instruction-following capability by explicitly aligning semantic instructions with spatial regions through spatial guidance during the denoising process, while the latter preserves features in unedited regions to maintain background consistency. To enable effective training, we construct a dedicated data pipeline to mitigate the scarcity of complex instruction-based image editing datasets, combining fine-grained automatic filtering via a powerful MLLM with rigorous human validation. Finally, to comprehensively evaluate complex instruction-based image editing, we introduce CIE-Bench, a new benchmark with two new evaluation metrics. Experimental results on CIE-Bench demonstrate that MCIE-E1 consistently outperforms previous state-of-the-art methods in both quantitative and qualitative assessments, achieving a 23.96% improvement in instruction compliance.

Problem

• instruction-based image editing
• complex instructions
• instruction compliance
• background consistency
• multimodal LLM

Innovation

• Multimodal LLM
• Complex Instruction Image Editing
• Spatial Guidance
• Background Consistency
• CIE-Bench