Multi-source Domain Adaptation for Panoramic Semantic Segmentation

2024-08-29

Information Fusion

AI Summary

To address the performance degradation of semantic segmentation on unlabeled real-world 360° panoramic images due to scarce annotations, this paper proposes an unsupervised multi-source-to-target domain adaptation framework to enhance model generalization. Methodologically, we pioneer the integration of cross-domain adaptation into panoramic segmentation, introducing three core components: (i) multi-source feature disentanglement via adversarial training, (ii) a panoramic spatial attention mechanism to preserve global structural coherence, and (iii) self-supervised depth-geometry consistency regularization to enforce geometric plausibility across domains. These jointly achieve both multi-source feature alignment and panoramic structure-aware consistency. Evaluated on PanoSUN and Stanford2D3D benchmarks, our method achieves an absolute mIoU improvement of 8.2% over state-of-the-art single-source and conventional multi-source adaptation approaches, demonstrating the effectiveness and advancement of the proposed panoramic structure-aware transfer paradigm.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Computer Vision: Segmentation
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs

Problem

• Panoramic Image Segmentation
• Learning Challenges
• Performance Limitations

Innovation

• DTA4PASS
• USM
• DGA