UDA4Inst: Unsupervised Domain Adaptation for Instance Segmentation

2024-05-15

AI Summary

To address the high annotation cost and scarcity of real-world data for instance segmentation in autonomous driving, alongside the limited performance of existing unsupervised domain adaptation (UDA) methods on this task, this paper proposes the first UDA framework for multi-source synthetic → multi-target real-world instance segmentation. Methodologically, it introduces semantic-category grouping training and bidirectional instance- and patch-level cross-domain image mixing, integrated with consistency regularization and pseudo-label refinement to significantly improve pseudo-label quality and model generalization. Built upon Mask R-CNN, our approach achieves 31.3 mAP on SYNTHIA→Cityscapes, setting a new state-of-the-art. Moreover, we present the first systematic evaluation on previously unreported cross-domain settings—UrbanSyn→Cityscapes and Synscapes→KITTI360—establishing reproducible benchmarks and methodological foundations for synthetic-to-real instance segmentation UDA.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Computer Vision: Segmentation
• Application Domains: Mobility, Driving & Flight

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Systems and Infrastructure for Web, Mobile and WoT: Web applications in cross-disciplinary domains and verticals such as mixed reality, smart cities, and digital health

Abstract

Instance segmentation is crucial for autonomous driving but is hindered by the lack of annotated real-world data due to expensive labeling costs. Unsupervised Domain Adaptation (UDA) offers a solution by transferring knowledge from labeled synthetic data to unlabeled real-world data. While UDA methods for synthetic to real-world domains (synth-to-real) show remarkable performance in tasks such as semantic segmentation and object detection, very few have been proposed for instance segmentation in vision-based autonomous driving. Moreover, existing methods rely on suboptimal baselines, which severely limits performance. We introduce extbf{UDA4Inst}, a powerful framework for synth-to-real UDA in instance segmentation. Our framework enhances instance segmentation through extit{Semantic Category Training} and extit{Bidirectional Mixing Training}. With the Semantic Category Training method, semantically related classes are grouped and trained separately, enabling the generation of higher-quality pseudo-labels and improved segmentation performance. We further propose a bidirectional cross-domain data mixing strategy that combines instance-wise and patch-wise mixing techniques to effectively utilize data from both source and target domains, producing realistic composite images that improve the model's generalization performance. Extensive experiments demonstrate the effectiveness of our methods. Our approach establishes a new state-of-the-art on the SYNTHIA->Cityscapes benchmark with mAP 31.3. Notably, we are the first to report results on multiple novel synth-to-real instance segmentation datasets, using UrbanSyn and Synscapes as source domains while Cityscapes and KITTI360 serve as target domains. Our code will be released soon.

Problem

• Instance Segmentation
• Unsupervised Domain Adaptation
• Data Annotation Cost

Innovation

• Unsupervised Domain Adaptation
• Instance Segmentation
• Data Mixing Strategy