Toward 360-Degree Indoor Panorama Editing via Tuning-Free Diffusion Model with Refocusing Cross-Attention

2026-06-11

AI Summary

This work addresses key limitations of zero-shot text-guided diffusion models in image editing—namely, prompt sensitivity, unintended edits in non-target regions, and failure to manipulate small or cluttered objects—by introducing FocusDiff, a novel framework that enables precise region-specific editing without fine-tuning and in a single forward pass. Given a target mask, FocusDiff employs a refocused cross-attention mechanism that selectively blurs non-editing regions to steer attention toward the intended area, while a context-preserving module maintains background consistency and global coherence. As the first approach integrating refocused attention with context preservation, FocusDiff significantly outperforms existing methods on the newly introduced LIMB benchmark across multiple criteria, including text alignment, background retention, editing accuracy, realism, and usability. The framework further demonstrates its practical potential by successfully extending to 360-degree indoor panoramic editing for virtual reality applications.

Abstract

Zero-shot text-guided diffusion has significantly advanced image editing; however, its practical usability remains constrained by three persistent challenges: prompt brittleness that requires meticulous prompt engineering, spillover edits that unintentionally affect non-target regions, and failures on small or cluttered objects caused by limited fine-grained supervision in training data. We propose FocusDiff (Target-Aware Refocusing for Tuning-Free Diffusion Editing), a tuning-free framework for precise and region-specific image manipulation based on refocusing cross-attention. Given a target region obtained through automated segmentation or manual selection, FocusDiff applies selective blurring to non-edit areas to guide attention toward the masked region while accurately transferring the object's identity, structure, and appearance to the edited output. Integrated context-preserving modules further ensure background fidelity and global coherence, enabling accurate edits from simple text prompts in a single pass. We also extend FocusDiff to 360-degree indoor panorama editing and demonstrate its effectiveness within virtual reality environments. Extensive experiments on our localized editing benchmark LIMB, comprising 30 multi-object images and 100 annotated examples including challenging small-object cases, show that FocusDiff outperforms existing zero-shot editors in text-image alignment and background preservation, achieving superior precision, photorealism, and usability. The project page is available at https://vdkhoi20.github.io/FocusDiff.

Problem

• prompt brittleness
• spillover edits
• small-object editing
• zero-shot image editing
• 360-degree panorama editing

Innovation

• tuning-free diffusion
• refocusing cross-attention
• region-specific editing
• 360-degree panorama editing
• zero-shot text-guided editing