Adversarial Domain Prompt Tuning and Generation for Single Domain Generalization

2026-06-19

AI Summary

This work addresses the challenge of single-source domain generalization by proposing a progressive adversarial prompt tuning framework, which, for the first time, leverages pre-trained text-to-image diffusion models for this task. Instead of handcrafted textual prompts, the method employs learnable abstract prompts that preserve semantic class information while generating diverse out-of-domain images to simulate the styles of unknown target domains. The framework jointly optimizes domain-invariant and domain-specific representations through an adversarial mechanism to enhance model robustness. Extensive experiments demonstrate that the proposed approach significantly outperforms existing single-domain generalization methods across multiple benchmarks, thereby validating the effectiveness of AIGC techniques in improving cross-domain generalization capabilities.

Abstract

Single domain generalization (SDG) aims to learn a robust model, which could perform well on many unseen domains while there is only one single domain available for training. One of the promising directions for achieving single-domain generalization is to generate out-of-domain (OOD) training data through data augmentation or image generation. Given the rapid advancements in AI-generated content (AIGC), this paper is the first to propose leveraging powerful pre-trained text-to-image (T2I) foundation models to create the training data. However, manually designing textual prompts to generate images for all possible domains is often impractical, and some domain characteristics may be too abstract to describe with words. To address these challenges, we propose a novel Progressive Adversarial Prompt Tuning (PAPT) framework for pre-trained diffusion models. Instead of relying on static textual domains, our approach learns two sets of abstract prompts as conditions for the diffusion model: one that captures domain-invariant category information and another that models domain-specific styles. This adversarial learning mechanism enables the T2I model to generate images in various domain styles while preserving key categorical features. Extensive experiments demonstrate the effectiveness of the proposed method, achieving superior performances to state-of-the-art single-domain generalization approaches.

Problem

• single domain generalization
• out-of-domain generation
• text-to-image generation
• domain adaptation
• adversarial learning

Innovation

• Adversarial Prompt Tuning
• Text-to-Image Generation
• Single Domain Generalization
• Diffusion Models
• Domain-Invariant Representation