Group-On: Boosting One-Shot Segmentation with Supportive Query

2024-04-18

arXiv.org

AI Summary

One-shot semantic segmentation suffers from performance degradation due to large intra- and inter-class variations in appearance and pose; existing methods rely on multiple annotated support images, incurring high labeling costs. This paper proposes a “query-as-support” co-optimization paradigm: within a batch, query images serve as mutual pseudo-support samples—eliminating the need for additional annotations. We design a lightweight Group-On Voting module to enable cross-query mask mutual enhancement. Built upon ASNet/HSNet, our framework establishes intra-batch query interaction through coarse prediction, pseudo-support construction, and voting-based fusion. On COCO-20i, our method achieves mIoU improvements of 8.21% and 7.46% over ASNet and HSNet, respectively. Notably, its one-shot performance surpasses most five-shot approaches, marking the first time that one-shot segmentation accuracy approaches that of multi-shot settings.

Technology Category

• Computer Vision: Segmentation
• Search and Optimization: Learning to Search
• Machine Learning: Semi-Supervised Learning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

One-shot semantic segmentation aims to segment query images given only ONE annotated support image of the same class. This task is challenging because target objects in the support and query images can be largely different in appearance and pose (i.e., intra-class variation). Prior works suggested that incorporating more annotated support images in few-shot settings boosts performances but increases costs due to additional manual labeling. In this paper, we propose a novel approach for ONE-shot semantic segmentation, called Group-On, which packs multiple query images in batches for the benefit of mutual knowledge support within the same category. Specifically, after coarse segmentation masks of the batch of queries are predicted, query-mask pairs act as pseudo support data to enhance mask predictions mutually, under the guidance of a simple Group-On Voting module. Comprehensive experiments on three standard benchmarks show that, in the ONE-shot setting, our Group-On approach significantly outperforms previous works by considerable margins. For example, on the COCO-20i dataset, we increase mIoU scores by 8.21% and 7.46% on ASNet and HSNet baselines, respectively. With only one support image, Group-On can be even competitive with the counterparts using 5 annotated support images.

Problem

• Addresses one-shot semantic segmentation with intra-class variation challenges
• Reduces manual labeling costs by using pseudo support data
• Enhances segmentation accuracy via mutual query-mask support

Innovation

• Batch processing of query images for mutual support
• Pseudo support data from query-mask pairs
• MoME module with multiple mask experts