LooseRoPE: Content-aware Attention Manipulation for Semantic Harmonization

2026-01-08

arXiv.org

AI Summary

This work addresses the inherent tension in prompt-free image editing between preserving the identity of a pasted object and achieving seamless semantic integration with the background. To resolve this, the authors propose LooseRoPE, a diffusion-based method that enables continuous, content-aware control of attention mechanisms through saliency-guided modulation of Rotary Position Embeddings (RoPE). By adaptively adjusting positional encoding without requiring textual prompts or complex user inputs, LooseRoPE effectively balances object identity fidelity with contextual coherence. Experimental results demonstrate that the method achieves high-fidelity, semantically harmonious image compositing directly from user-provided crop-and-paste operations, significantly outperforming existing prompt-free editing approaches.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Natural Language Processing: Prompt Engineering / Prompting
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

Recent diffusion-based image editing methods commonly rely on text or high-level instructions to guide the generation process, offering intuitive but coarse control. In contrast, we focus on explicit, prompt-free editing, where the user directly specifies the modification by cropping and pasting an object or sub-object into a chosen location within an image. This operation affords precise spatial and visual control, yet it introduces a fundamental challenge: preserving the identity of the pasted object while harmonizing it with its new context. We observe that attention maps in diffusion-based editing models inherently govern whether image regions are preserved or adapted for coherence. Building on this insight, we introduce LooseRoPE, a saliency-guided modulation of rotational positional encoding (RoPE) that loosens the positional constraints to continuously control the attention field of view. By relaxing RoPE in this manner, our method smoothly steers the model's focus between faithful preservation of the input image and coherent harmonization of the inserted object, enabling a balanced trade-off between identity retention and contextual blending. Our approach provides a flexible and intuitive framework for image editing, achieving seamless compositional results without textual descriptions or complex user input.

Problem

• image editing
• semantic harmonization
• identity preservation
• contextual blending
• prompt-free editing

Innovation

• LooseRoPE
• rotational positional encoding
• attention manipulation
• prompt-free editing
• semantic harmonization