Prmpt2Adpt: Prompt-Based Zero-Shot Domain Adaptation for Resource-Constrained Environments

2025-06-20

AI Summary

To address the deployment challenges of unsupervised domain adaptation (UDA) for object detection in resource-constrained scenarios (e.g., UAVs), this paper proposes a zero-shot, source-data-free cross-domain detection framework. Methodologically, it introduces a prompt-driven instance normalization (PIN) mechanism—enabling semantic-aware feature alignment using only a few source images and natural language prompts—and adopts an efficient paradigm comprising a frozen large vision model backbone, lightweight detector head fine-tuning, and pseudo-label distillation, where a distilled CLIP visual encoder serves as the compact student model. Evaluated on the MDS-A benchmark, the method achieves state-of-the-art accuracy, accelerates domain adaptation by 7× and inference by 5×, while significantly reducing memory footprint and computational overhead.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Computer Vision: Diffusion Models for Vision
• Application Domains: Mobility, Driving & Flight

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Unsupervised Domain Adaptation (UDA) is a critical challenge in real-world vision systems, especially in resource-constrained environments like drones, where memory and computation are limited. Existing prompt-driven UDA methods typically rely on large vision-language models and require full access to source-domain data during adaptation, limiting their applicability. In this work, we propose Prmpt2Adpt, a lightweight and efficient zero-shot domain adaptation framework built around a teacher-student paradigm guided by prompt-based feature alignment. At the core of our method is a distilled and fine-tuned CLIP model, used as the frozen backbone of a Faster R-CNN teacher. A small set of low-level source features is aligned to the target domain semantics-specified only through a natural language prompt-via Prompt-driven Instance Normalization (PIN). These semantically steered features are used to briefly fine-tune the detection head of the teacher model. The adapted teacher then generates high-quality pseudo-labels, which guide the on-the-fly adaptation of a compact student model. Experiments on the MDS-A dataset demonstrate that Prmpt2Adpt achieves competitive detection performance compared to state-of-the-art methods, while delivering up to 7x faster adaptation and 5x faster inference speed using few source images-making it a practical and scalable solution for real-time adaptation in low-resource domains.

Problem

• Lightweight zero-shot domain adaptation for resource-constrained environments
• Prompt-based feature alignment without full source data access
• Efficient real-time adaptation with minimal source images

Innovation

• Lightweight zero-shot domain adaptation framework
• Prompt-driven feature alignment with CLIP
• Teacher-student model for efficient adaptation