Is Domain Adaptation Always Helpful? A Frozen-Backbone Study of Cross-Domain Sentiment Transfer

2026-07-07

AI Summary

This study investigates whether explicit domain adaptation consistently improves cross-domain sentiment analysis performance when the backbone of a pretrained language model is frozen. We train lightweight MLP adapters on frozen backbones of varying scales and domain specialization—including Qwen3-Embedding (0.6B/4B/8B), RoBERTa-base, and FinBERT—and evaluate them with alignment strategies such as DANN, MMD, and supervised contrastive learning (SCL) on tasks transferring from consumer reviews to movie reviews and financial news. Our findings reveal that the efficacy of domain adaptation heavily depends on whether the backbone already encodes target-domain knowledge. Adversarial alignment (e.g., DANN) can degrade performance on domain-specialized backbones like FinBERT, whereas SCL proves more robust. While adaptation yields negligible gains on SST-2, it substantially recovers performance for smaller general-purpose backbones on financial subsets.

Abstract

Sentiment analysis with frozen pre-trained language model (PLM) backbones has become a common paradigm, yet the practical benefit of explicit domain adaptation remains unclear, particularly when backbones encode varying degrees of target-domain knowledge. We present a preliminary case study evaluating a controlled family of frozen embedding backbones (Qwen3-Embedding 0.6B, 4B, 8B), alongside RoBERTa-base and FinBERT. We train a lightweight MLP adapter on consumer reviews using Domain-Adversarial Neural Networks (DANN), Maximum Mean Discrepancy (MMD), and Supervised Contrastive Learning (SCL), and evaluate transfer to movie reviews (SST-2) and a heavily restricted subset of financial news (Financial PhraseBank). Within this constrained sample, we observe two distinct transfer patterns. On SST-2, domain adaptation provides negligible gain regardless of scale. On the financial subset, explicit domain adaptation appears to recover substantial performance for small general-purpose backbones. Notably, we find that adversarial alignment (DANN) is associated with degraded performance for domain-specialized backbones like FinBERT, consistent with erosion of pre-existing domain-specific structure, whereas supervised contrastive loss appears to preserve it. These preliminary findings suggest that the efficacy of explicit domain adaptation is highly contingent on whether the frozen backbone already possesses target-domain coverage.

Problem

• domain adaptation
• sentiment analysis
• frozen backbone
• cross-domain transfer
• pre-trained language models

Innovation

• domain adaptation
• frozen backbone
• sentiment transfer
• supervised contrastive learning
• pre-trained language models