Target Semantics Clustering via Text Representations for Robust Universal Domain Adaptation

2025-04-11

AAAI Conference on Artificial Intelligence

AI Summary

UniDA confronts dual challenges: domain shift and unknown-class shift. Existing methods estimate target-domain semantic centers in continuous image spaces, suffering from noise sensitivity and difficulty in determining optimal cluster counts, thereby yielding non-robust alignment. This paper pioneers the migration of target-domain semantic clustering into the discrete, semantically grounded text embedding space of a frozen vision-language model. We propose UniMS, a unified scoring function that jointly models shared-class alignment and private-class discovery. Our framework integrates greedy text-space search, gradient-driven fine-tuning, information-maximization optimization, and an open-set discriminative mechanism. Evaluated across four standard UniDA benchmarks and four category-shift scenarios, our approach achieves state-of-the-art performance, significantly improving cross-domain robustness and private-class identification accuracy.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Computer Vision: Adversarial Attacks & Robustness
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Universal Domain Adaptation (UniDA) focuses on transferring source domain knowledge to the target domain under both domain shift and unknown category shift. Its main challenge lies in identifying common class samples and aligning them. Current methods typically obtain target domain semantics centers from an unconstrained continuous image representation space. Due to domain shift and the unknown number of clusters, these centers often result in complex and less robust alignment algorithm. In this paper, based on vision-language models, we search for semantic centers in a semantically meaningful and discrete text representation space. The constrained space ensures almost no domain bias and appropriate semantic granularity for these centers, enabling a simple and robust adaptation algorithm. Specifically, we propose TArget Semantics Clustering (TASC) via Text Representations, which leverages information maximization as a unified objective and involves two stages. First, with the frozen encoders, a greedy search-based framework is used to search for an optimal set of text embeddings to represent target semantics. Second, with the search results fixed, encoders are refined based on gradient descent, simultaneously achieving robust domain alignment and private class clustering. Additionally, we propose Universal Maximum Similarity (UniMS), a scoring function tailored for detecting open-set samples in UniDA. Experimentally, we evaluate the universality of UniDA algorithms under four category shift scenarios. Extensive experiments on four benchmarks demonstrate the effectiveness and robustness of our method, which has achieved state-of-the-art performance.

Problem

• Identifying common class samples under domain shift
• Aligning target domain semantics with constrained text space
• Detecting open-set samples in Universal Domain Adaptation

Innovation

• Uses text representation space for clustering
• Leverages vision-language models for semantics
• Proposes Universal Maximum Similarity scoring