🤖 AI Summary
To address the performance bottleneck of interactive occlusion boundary estimation (IOBE) caused by the scarcity of real-world occlusion boundary (OB) annotations, this paper proposes DNMMSI—the first interactive IOBE method supporting multi-stroke interventions—and introduces two novel benchmarks: OB-FUTURE, a geometrically precise synthetic benchmark built upon our self-developed differentiable tool Mesh2OB, and OB-LabName, a high-fidelity real-world benchmark comprising 120 images with pixel-level ground truth. Key contributions include: (i) establishing the first formal interactive OB estimation paradigm; (ii) introducing the first end-to-end training framework leveraging geometry-differentiable synthetic data, eliminating the need for domain adaptation; and (iii) releasing the first high-fidelity real OB benchmark alongside a fully open-sourced toolchain. Experiments demonstrate that DNMMSI, trained solely on synthetic data, significantly outperforms state-of-the-art fully automatic methods, while OB-LabName achieves superior annotation accuracy compared to existing benchmarks.
📝 Abstract
Occlusion boundaries (OBs) geometrically localize the occlusion events in a 2D image, and contain useful information for addressing various scene understanding problems. To advance their study, we have led the investigation in the following three aspects. Firstly, we have studied interactive estimation of OBs, which is the first in the literature, and proposed an efficient deep-network-based method using multiple-scribble intervention, named DNMMSI, which significantly improves the performance over the state-of-the-art fully-automatic methods. Secondly, we propose to exploit the synthetic benchmark for the training, thanks to the particularity that OBs are determined geometrically and unambiguously from the 3D scene. To this end, we have developed an efficient tool, named Mesh2OB, for the automatic generation of 2D images together with their ground-truth OBs, using which we have constructed a synthetic benchmark, named OB-FUTURE. Abundant experimental results demonstrate that leveraging such a synthetic benchmark for training achieves promising performance, even without the use of domain adaptation techniques. Finally, to achieve a more compelling and robust evaluation in OB-related research, we have created a real-world benchmark OB-LabName, consisting of 120 high-resolution images together with their ground-truth OBs, with precision surpassing that of previous benchmarks. We will release DNMMSI with pre-trained parameters, Mesh2OB, OB-FUTURE, and OB-LabName to support further research.