🤖 AI Summary
To address inaccurate personalized instance segmentation in dense images caused by severe occlusion, scale variation, and cluttered backgrounds, this paper proposes the first end-to-end, training-free, model-agnostic one-shot segmentation framework. Our method comprises two core components: (1) a density-map-based Instance Detection Module (IDM) for coarse-grained localization; and (2) a Point Prompt Selection Module (PPSM) coupled with a feedback-driven exemplar self-selection mechanism, which jointly generate highly discriminative instance-level point prompts and iteratively refine the density map. Evaluated on our newly constructed benchmark PerSense-D, the framework achieves 71.61% mIoU—surpassing the state-of-the-art by up to 47.16%. This substantial improvement demonstrates significantly enhanced robustness in challenging, unconstrained real-world scenarios.
📝 Abstract
Leveraging large-scale pre-training, vision foundational models showcase notable performance benefits. Recent segmentation algorithms for natural scenes have advanced significantly. However, existing models still struggle to automatically segment personalized instances in dense and crowded scenarios, where severe occlusions, scale variations, and background clutter pose a challenge to accurately delineate densely packed instances of the target object. To address this, we propose PerSense, an end-to-end, training-free, and model-agnostic one-shot framework for Personalized instance Segmentation in dense images. We develop a new baseline capable of automatically generating instance-level point prompts via proposing a novel Instance Detection Module (IDM) that leverages density maps, encapsulating spatial distribution of objects in an image. To mitigate false positives within generated point prompts, we design Point Prompt Selection Module (PPSM). Both IDM and PPSM transform density maps into personalized precise point prompts for instance-level segmentation and offer a seamless integration in our model-agnostic framework. We also introduce a feedback mechanism which enables PerSense to improve the accuracy of density maps by automating the exemplar selection process for density map generation. To promote algorithmic advances and effective tools for this relatively underexplored task, we introduce PerSense-D, a diverse dataset exclusive to personalized instance segmentation in dense images. Our extensive experiments establish PerSense superiority in dense scenarios by achieving an mIoU of 71.61% on PerSense-D, outperforming recent SOTA models by significant margins of +47.16%, +42.27%, +8.83%, and +5.69%. Additionally, our qualitative findings demonstrate the adaptability of our framework to images captured in-the-wild.