🤖 AI Summary
Severe occlusion critically undermines the robustness of panoptic segmentation models. Method: We introduce COCO-Occ—the first occlusion-aware panoptic segmentation benchmark—constructed by annotating three levels of perceptual occlusion severity on the COCO dataset to systematically quantify occlusion’s impact on performance. We further propose Occlusion-Aware Contrastive Learning (OACL), a novel framework that explicitly models occlusion severity to learn occlusion-robust representations. Contribution/Results: Fine-tuning state-of-the-art panoptic segmentation models on COCO-Occ with OACL yields significant performance gains under medium and heavy occlusion, achieving new SOTA results. Experiments demonstrate that baseline models suffer sharp performance degradation as occlusion severity increases, whereas OACL effectively mitigates this deterioration. This work establishes the first graded occlusion benchmark for panoptic segmentation and introduces a principled paradigm for occlusion-robust visual understanding.
📝 Abstract
To help address the occlusion problem in panoptic segmentation and image understanding, this paper proposes a new large-scale dataset named COCO-OLAC (COCO Occlusion Labels for All Computer Vision Tasks), which is derived from the COCO dataset by manually labelling images into three perceived occlusion levels. Using COCO-OLAC, we systematically assess and quantify the impact of occlusion on panoptic segmentation on samples having different levels of occlusion. Comparative experiments with SOTA panoptic models demonstrate that the presence of occlusion significantly affects performance, with higher occlusion levels resulting in notably poorer performance. Additionally, we propose a straightforward yet effective method as an initial attempt to leverage the occlusion annotation using contrastive learning to render a model that learns a more robust representation capturing different severities of occlusion. Experimental results demonstrate that the proposed approach boosts the performance of the baseline model and achieves SOTA performance on the proposed COCO-OLAC dataset.