ProMi: An Efficient Prototype-Mixture Baseline for Few-Shot Segmentation with Bounding-Box Annotations

2025-05-18

AI Summary

To address the challenge of expensive pixel-level annotations in few-shot image segmentation for robotic applications, this paper proposes a training-agnostic prototype mixing framework supervised solely by bounding boxes. The method models the background class as a Gaussian mixture distribution prototype—novelly integrated with foreground prototypes—to form a learnable dual-prototype structure compatible with coarse-grained supervision. Leveraging ResNet-50 for feature extraction, it performs zero-shot segmentation in the feature space via distance-based metric learning, eliminating the need for fine-tuning. Lightweight and efficient, the approach achieves state-of-the-art performance on standard benchmarks including PASCAL-5i and COCO-20i, significantly outperforming existing bounding-box-supervised methods. Extensive experiments on real-world mobile robot tasks further demonstrate its strong generalization capability and robustness under practical deployment conditions.

Technology Category

• Computer Vision: Segmentation
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multi-instance/Multi-view Learning

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

In robotics applications, few-shot segmentation is crucial because it allows robots to perform complex tasks with minimal training data, facilitating their adaptation to diverse, real-world environments. However, pixel-level annotations of even small amount of images is highly time-consuming and costly. In this paper, we present a novel few-shot binary segmentation method based on bounding-box annotations instead of pixel-level labels. We introduce, ProMi, an efficient prototype-mixture-based method that treats the background class as a mixture of distributions. Our approach is simple, training-free, and effective, accommodating coarse annotations with ease. Compared to existing baselines, ProMi achieves the best results across different datasets with significant gains, demonstrating its effectiveness. Furthermore, we present qualitative experiments tailored to real-world mobile robot tasks, demonstrating the applicability of our approach in such scenarios. Our code: https://github.com/ThalesGroup/promi.

Problem

• Few-shot segmentation with minimal training data
• Replacing pixel-level labels with bounding-box annotations
• Efficient prototype-mixture method for coarse annotations

Innovation

• Uses bounding-box annotations instead of pixel-level labels
• Treats background class as mixture of distributions
• Simple, training-free, and effective few-shot segmentation