P2Object: Single Point Supervised Object Detection and Instance Segmentation

2025-04-10

AI Summary

Under point-supervision, object detection and instance segmentation performance lags significantly behind fully supervised methods, primarily due to inaccurate pseudo-box localization, boundary truncation, and excessive background inclusion caused by discrete bounding-box sampling in conventional multiple-instance learning. Method: We propose a “point→box→mask” continuous perception paradigm: (i) the first instance-level proposal bag construction mechanism; (ii) a discrete-to-continuous optimization pathway enabling pixel-level object awareness; and (iii) a boundary self-prediction module unifying joint detection and segmentation optimization. Our framework—comprising P2BNet, P2BNet++ (with continuous proposal sampling), and P2MNet—is an end-to-end pixel prediction architecture integrating low-level features and spatial cues. Results: Evaluated on COCO, VOC, SBD, and Cityscapes, our approach substantially outperforms existing point-supervised methods, achieving significant mAP gains and markedly narrowing the performance gap with fully supervised baselines.

Technology Category

• Computer Vision: Object Detection & Categorization
• Machine Learning: Multi-instance/Multi-view Learning
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Object recognition using single-point supervision has attracted increasing attention recently. However, the performance gap compared with fully-supervised algorithms remains large. Previous works generated class-agnostic extbf{ extit{proposals in an image}} offline and then treated mixed candidates as a single bag, putting a huge burden on multiple instance learning (MIL). In this paper, we introduce Point-to-Box Network (P2BNet), which constructs balanced extbf{ extit{instance-level proposal bags}} by generating proposals in an anchor-like way and refining the proposals in a coarse-to-fine paradigm. Through further research, we find that the bag of proposals, either at the image level or the instance level, is established on discrete box sampling. This leads the pseudo box estimation into a sub-optimal solution, resulting in the truncation of object boundaries or the excessive inclusion of background. Hence, we conduct a series exploration of discrete-to-continuous optimization, yielding P2BNet++ and Point-to-Mask Network (P2MNet). P2BNet++ conducts an approximately continuous proposal sampling strategy by better utilizing spatial clues. P2MNet further introduces low-level image information to assist in pixel prediction, and a boundary self-prediction is designed to relieve the limitation of the estimated boxes. Benefiting from the continuous object-aware extbf{ extit{pixel-level perception}}, P2MNet can generate more precise bounding boxes and generalize to segmentation tasks. Our method largely surpasses the previous methods in terms of the mean average precision on COCO, VOC, SBD, and Cityscapes, demonstrating great potential to bridge the performance gap compared with fully supervised tasks.

Problem

• Bridges performance gap in single-point supervised object detection
• Improves pseudo box estimation via discrete-to-continuous optimization
• Enhances pixel-level perception for precise segmentation tasks

Innovation

• Anchor-like proposal generation for balanced instance-level bags
• Discrete-to-continuous optimization via spatial clues
• Pixel-level perception for precise boundary prediction