Physics-Aware Video Instance Removal Benchmark

2026-04-07

AI Summary

Existing video instance removal methods often neglect physical consistency—such as shadow and lighting interactions—leading to background distortions. This work proposes PVIR, the first benchmark for video instance removal that explicitly emphasizes physical causal consistency. PVIR comprises 95 high-quality videos annotated with instance-level masks and textual removal prompts, and is partitioned into Simple and Hard subsets to reflect varying levels of physical interaction complexity. Using a decoupled human evaluation protocol, the study systematically assesses current methods across three dimensions: instruction adherence, rendering quality, and editing exclusivity. Results indicate that PISCO-Removal and UniVideo achieve the best overall performance, yet all approaches exhibit significant performance degradation on the Hard subset, underscoring that accurately restoring complex physical effects remains a critical challenge in video editing.

Abstract

Video Instance Removal (VIR) requires removing target objects while maintaining background integrity and physical consistency, such as specular reflections and illumination interactions. Despite advancements in text-guided editing, current benchmarks primarily assess visual plausibility, often overlooking the physical causalities, such as lingering shadows, triggered by object removal. We introduce the Physics-Aware Video Instance Removal (PVIR) benchmark, featuring 95 high-quality videos annotated with instance-accurate masks and removal prompts. PVIR is partitioned into Simple and Hard subsets, the latter explicitly targeting complex physical interactions. We evaluate four representative methods, PISCO-Removal, UniVideo, DiffuEraser, and CoCoCo, using a decoupled human evaluation protocol across three dimensions to isolate semantic, visual, and spatial failures: instruction following, rendering quality, and edit exclusivity. Our results show that PISCO-Removal and UniVideo achieve state-of-the-art performance, while DiffuEraser frequently introduces blurring artifacts and CoCoCo struggles significantly with instruction following. The persistent performance drop on the Hard subset highlights the ongoing challenge of recovering complex physical side effects.

Problem

• Video Instance Removal
• Physical Consistency
• Specular Reflections
• Illumination Interactions
• Shadow Artifacts

Innovation

• Physics-Aware Editing
• Video Instance Removal
• Physical Consistency
• Benchmark Dataset
• Decoupled Evaluation