Multi-Modal Guided Multi-Source Domain Adaptation for Object Detection

2026-05-13

AI Summary

This work addresses the performance degradation in multi-source unsupervised domain adaptive object detection caused by distributional shifts between target domains and training data. To this end, the authors propose MS-DePro, a novel approach that introduces depth map–guided generation of domain-invariant region proposals for the first time. By integrating learnable text embeddings, MS-DePro enables multimodal (depth and text) guided prompt learning, thereby overcoming the limitations of conventional methods that rely solely on RGB images. The framework synergistically combines depth estimation, multimodal feature alignment, and domain adaptation mechanisms to achieve state-of-the-art performance across multiple benchmarks. Ablation studies further confirm the effectiveness of each proposed component.

Abstract

General object detection (OD) struggles to detect objects in the target domain that differ from the training distribution. To address this, recent studies demonstrate that training from multiple source domains and explicitly processing them separately for multi-source domain adaptation (MSDA) outperforms blending them for unsupervised domain adaptation (UDA). However, existing MSDA methods learn domain-agnostic features from domain-specific RGB images while preserving domain-specific information from the domain-agnostic feature map. To address this, we propose MS-DePro: Multi-Source Detector with Depth and Prompt, composed of (1) depth-guided localization and (2) multi-modal guided prompt learning. We leverage domain-agnostic input modalities, namely depth maps and text, to encode domain-agnostic characteristics. Specifically, we utilize depth maps to generate domain-agnostic region proposals for localization and integrate multi-modal features to align learnable text embeddings for classification. MS-DePro achieves state-of-the-art performance on MSDA benchmarks, and comprehensive ablations demonstrate the effectiveness of our contributions. Our code is available on https://github.com/sejong-rcv/Multi-Modal-Guided-Multi-Source-Domain-Adaptation-for-Object-Detection.

Problem

• object detection
• multi-source domain adaptation
• domain shift
• domain-agnostic features
• unsupervised domain adaptation

Innovation

• multi-source domain adaptation
• depth-guided localization
• multi-modal prompt learning
• object detection
• domain-agnostic features