PANDORA: Pixel-wise Attention Dissolution and Latent Guidance for Zero-Shot Object Removal

2026-03-29

AI Summary

This work addresses the challenges of object removal in natural images—namely semantic inconsistency, structural degradation of background content, and difficulties in handling multiple objects—by introducing a zero-shot object removal framework. The proposed method operates directly on pretrained text-to-image diffusion models without requiring fine-tuning, user prompts, or test-time optimization, achieving high-quality background reconstruction. Leveraging pixel-wise attention disentanglement and locally decoupled attention guidance, it enables, for the first time in a zero-shot setting, precise, non-rigid, prompt-free, and scalable one-step removal of multiple objects, substantially enhancing foreground-background disentanglement. Experimental results demonstrate that the approach outperforms state-of-the-art methods in both visual fidelity and semantic plausibility.

Abstract

Removing objects from natural images is challenging due to difficulty of synthesizing semantically coherent content while preserving background integrity. Existing methods often rely on fine-tuning, prompt engineering, or inference-time optimization, yet still suffer from texture inconsistency, rigid artifacts, weak foreground-background disentanglement, and poor scalability for multi-object removal. We propose a novel zero-shot object removal framework, namely PANDORA, that operates directly on pre-trained text-to-image diffusion models, requiring no fine-tuning, prompts, or optimization. We propose Pixel-wise Attention Dissolution to remove object by nullifying the most correlated attention keys for masked pixels, effectively eliminating the object from self-attention flow and allowing background context to dominate reconstruction. We further introduce Localized Attentional Disentanglement Guidance to steer denoising toward latent manifolds favorable to clean object removal. Together, these components enable precise, non-rigid, prompt-free, and scalable multi-object erasure in a single pass. Experiments demonstrate superior visual fidelity and semantic plausibility compared to state-of-the-art methods. The project page is available at https://vdkhoi20.github.io/PANDORA.

Problem

• object removal
• semantic coherence
• background integrity
• multi-object removal
• texture inconsistency

Innovation

• Pixel-wise Attention Dissolution
• Localized Attentional Disentanglement Guidance
• Zero-Shot Object Removal
• Diffusion Models
• Attention Manipulation