BoxSeg: Quality-Aware and Peer-Assisted Learning for Box-supervised Instance Segmentation

2025-04-07

AI Summary

This paper addresses the challenge of achieving high-quality instance segmentation using only bounding box annotations. To this end, we propose BoxSeg, a novel framework featuring a Quality-Aware Module (QAM) that dynamically evaluates and refines pseudo-mask confidence, and a Peer-Assisted Copy-Paste (PC) mechanism that leverages high-confidence pseudo-masks to rectify low-confidence ones. Integrated within a teacher-student paradigm and augmented with a multi-mask complementarity strategy, these components collectively enhance the robustness of pseudo-labels. Our key contributions are: (1) the first quality-adaptive pseudo-mask optimization mechanism specifically designed for box-supervised instance segmentation; and (2) transferable, plug-and-play QAM and PC modules that significantly outperform existing state-of-the-art methods on COCO, while also boosting the performance of other box-supervised models without architectural modifications.

Technology Category

• Computer Vision: Segmentation
• Search and Optimization: Learning to Search
• Machine Learning: Multi-instance/Multi-view Learning

Application Category

• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Box-supervised instance segmentation methods aim to achieve instance segmentation with only box annotations. Recent methods have demonstrated the effectiveness of acquiring high-quality pseudo masks under the teacher-student framework. Building upon this foundation, we propose a BoxSeg framework involving two novel and general modules named the Quality-Aware Module (QAM) and the Peer-assisted Copy-paste (PC). The QAM obtains high-quality pseudo masks and better measures the mask quality to help reduce the effect of noisy masks, by leveraging the quality-aware multi-mask complementation mechanism. The PC imitates Peer-Assisted Learning to further improve the quality of the low-quality masks with the guidance of the obtained high-quality pseudo masks. Theoretical and experimental analyses demonstrate the proposed QAM and PC are effective. Extensive experimental results show the superiority of our BoxSeg over the state-of-the-art methods, and illustrate the QAM and PC can be applied to improve other models.

Problem

• Achieve instance segmentation using only box annotations
• Improve pseudo mask quality with noise reduction
• Enhance low-quality masks via peer-assisted learning

Innovation

• Quality-Aware Module for high-quality pseudo masks
• Peer-assisted Copy-paste improves low-quality masks
• Box-supervised learning reduces noisy mask effects