LOCATEdit: Graph Laplacian Optimized Cross Attention for Localized Text-Guided Image Editing

2025-03-27

AI Summary

In text-guided image editing, existing diffusion-based cross-attention methods suffer from semantic misalignment, leading to inaccurate target localization, spatial inconsistency, and structural distortion. To address this, we propose a graph Laplacian-regularized cross-attention mechanism: first, self-attention constructs a patch-wise relational graph encoding local geometric and semantic similarity; second, the graph Laplacian operator imposes smoothness constraints on the cross-attention map, enhancing boundary coherence of edited regions and background fidelity. This work is the first to integrate structural priors from graph neural networks into the attention mechanism of diffusion models. Evaluated on PIE-Bench, our method significantly outperforms state-of-the-art approaches, achieving superior editing accuracy and visual quality. It effectively suppresses artifacts while preserving global structural integrity.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Graph-based Machine Learning
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Text-guided image editing aims to modify specific regions of an image according to natural language instructions while maintaining the general structure and the background fidelity. Existing methods utilize masks derived from cross-attention maps generated from diffusion models to identify the target regions for modification. However, since cross-attention mechanisms focus on semantic relevance, they struggle to maintain the image integrity. As a result, these methods often lack spatial consistency, leading to editing artifacts and distortions. In this work, we address these limitations and introduce LOCATEdit, which enhances cross-attention maps through a graph-based approach utilizing self-attention-derived patch relationships to maintain smooth, coherent attention across image regions, ensuring that alterations are limited to the designated items while retaining the surrounding structure. method consistently and substantially outperforms existing baselines on PIE-Bench, demonstrating its state-of-the-art performance and effectiveness on various editing tasks. Code can be found on https://github.com/LOCATEdit/LOCATEdit/

Problem

• Improves text-guided image editing accuracy
• Reduces artifacts in cross-attention based methods
• Enhances spatial consistency in localized edits

Innovation

• Graph Laplacian optimizes cross-attention maps
• Self-attention patch relationships ensure coherence
• Maintains image integrity while editing regions