Language-Induced Priors for Domain Adaptation

2026-05-13

AI Summary

This work addresses the challenge of negative transfer in domain adaptation when target-domain data are scarce, a setting where conventional methods struggle to identify relevant source domains. The authors propose the Language-Induced Prior (LIP) framework, which uniquely leverages textual descriptions of the target domain—processed through a pretrained large language model—to construct semantic priors for domain adaptation. These priors are integrated into an Expectation-Maximization algorithm to enable dynamic source-domain selection under cold-start conditions. Theoretically, LIP achieves near-optimal performance when the prior is accurate and guarantees asymptotic consistency of estimators even with arbitrary priors. Empirical results across Gaussian estimation, C-MAPSS degradation prediction, and MuJoCo control tasks demonstrate that LIP substantially mitigates negative transfer and enhances cross-domain generalization.

Abstract

Domain adaptation faces a fundamental paradox in the cold-start regime. When target data is scarce, statistical methods fail to distinguish relevant source domains from irrelevant ones, which often leads to negative transfer. In this paper, we address this challenge by leveraging expert textual descriptions of the target domain, a resource that is often available but overlooked. We propose a probabilistic framework that translates these semantic descriptions into a choice model, namely a Language-Induced Prior (LIP), that learns the preferences from a pretrained Large Language Model (LLM). The LIP is then integrated into an Expectation-Maximization algorithm to identify source relevance. Methodologically, this framework is compatible with any parametric model where a likelihood is available. It allows the LIP to guide the selection of sources when target signals are weak, while gradually refining these choices as samples accumulate. Theoretically, we prove that the estimator roughly matches an oracle cold-start MSE under a correct prior, while remaining asymptotically consistent regardless of the quality of the LIP. Empirically, we validated the framework on a descriptive (Gaussian estimation), a predictive (C-MAPSS dataset), and a prescriptive task (MuJoCo hopper).

Problem

• domain adaptation
• cold-start
• negative transfer
• source selection
• target data scarcity

Innovation

• Language-Induced Prior
• Domain Adaptation
• Cold-Start
• Large Language Model
• Expectation-Maximization