Knowledge Consultation for Semi-Supervised Semantic Segmentation

2025-03-12

AI Summary

To address the high reliance on labeled data and insufficient exploitation of unlabeled data in semi-supervised semantic segmentation, this paper proposes SegKC, a knowledge-consulting co-training paradigm. SegKC innovatively integrates knowledge distillation into a cross-pseudo-supervision framework, employing two heterogeneous CNN backbones to enable structured knowledge complementarity—beyond mere label exchange—and introduces a knowledge-consulting loss, consistency regularization, and a lightweight feature alignment module. On Pascal VOC, SegKC achieves 87.1% mIoU using only 25% of labeled data and 89.2% with 50%, approaching fully supervised performance (89.8%). Comparable gains are observed on Cityscapes, while maintaining a compact model size. This work elucidates an effective mechanism for cross-model knowledge transfer within deep co-training frameworks.

Technology Category

• Machine Learning: Semi-Supervised Learning
• Computer Vision: Segmentation
• Data Mining & Knowledge Management: Semantic Web

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Semi-Supervised Semantic Segmentation reduces reliance on extensive annotations by using unlabeled data and state-of-the-art models to improve overall performance. Despite the success of deep co-training methods, their underlying mechanisms remain underexplored. This work revisits Cross Pseudo Supervision with dual heterogeneous backbones and introduces Knowledge Consultation (SegKC) to further enhance segmentation performance. The proposed SegKC achieves significant improvements on Pascal and Cityscapes benchmarks, with mIoU scores of 87.1%, 89.2%, and 89.8% on Pascal VOC with the 1/4, 1/2, and full split partition, respectively, while maintaining a compact model architecture.

Problem

• Enhance semi-supervised semantic segmentation performance
• Explore mechanisms of deep co-training methods
• Introduce Knowledge Consultation for improved segmentation

Innovation

• Cross Pseudo Supervision with dual backbones
• Knowledge Consultation (SegKC) for enhanced segmentation
• Compact model architecture with high mIoU scores