Stein Discrepancy for Unsupervised Domain Adaptation

2025-02-05

AI Summary

In unsupervised domain adaptation (UDA) with abundant source but scarce target samples, existing methods based on Wasserstein distance or maximum mean discrepancy (MMD) suffer from degraded performance due to their reliance on sufficient target data. Method: This paper introduces Stein discrepancy—the first such application in UDA—leveraging the score function of the target distribution to construct an asymmetric, low-sample-sensitive domain alignment mechanism that alleviates strong dependence on target sample size. We theoretically derive a generalization error upper bound and propose both non-kernelized and kernelized Stein discrepancy optimization frameworks, integrated with differentiable score function estimation for deep feature alignment. Results: Extensive experiments on multiple UDA benchmarks under low-target-sample settings demonstrate consistent superiority over Wasserstein distance, MMD, and other mainstream approaches, achieving average accuracy gains of 2.1–5.7%.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Computer Vision: Adversarial Attacks & Robustness
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Unsupervised domain adaptation (UDA) leverages information from a labeled source dataset to improve accuracy on a related but unlabeled target dataset. A common approach to UDA is aligning representations from the source and target domains by minimizing the distance between their data distributions. Previous methods have employed distances such as Wasserstein distance and maximum mean discrepancy. However, these approaches are less effective when the target data is significantly scarcer than the source data. Stein discrepancy is an asymmetric distance between distributions that relies on one distribution only through its score function. In this paper, we propose a novel ac{uda} method that uses Stein discrepancy to measure the distance between source and target domains. We develop a learning framework using both non-kernelized and kernelized Stein discrepancy. Theoretically, we derive an upper bound for the generalization error. Numerical experiments show that our method outperforms existing methods using other domain discrepancy measures when only small amounts of target data are available.

Problem

• Unsupervised domain adaptation with Stein discrepancy
• Addressing scarcity of target data
• Improving accuracy on unlabeled datasets

Innovation

• Uses Stein discrepancy measure
• Develops non-kernelized and kernelized framework
• Improves accuracy with scarce target data