More Than Efficiency: Embedding Compression Improves Domain Adaptation in Dense Retrieval

2026-01-20

AI Summary

This work addresses the performance degradation of dense retrieval models under domain shift, a challenge exacerbated by the high cost of labeled data and retraining in conventional domain adaptation approaches. The authors propose a lightweight, annotation-free adaptation method that requires neither fine-tuning nor additional supervision. By applying Principal Component Analysis (PCA) to compress query embeddings, the method preserves domain-discriminative features while filtering out non-essential components. Evaluated across nine retrievers and fourteen MTEB datasets, this approach improves NDCG@10 in 75.4% of model–dataset combinations through query embedding compression alone. The technique simultaneously enhances computational efficiency and cross-domain retrieval effectiveness, offering a novel paradigm for efficient domain adaptation in dense retrieval systems.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Data Mining & Knowledge Management: Data Compression
• Computer Vision: Image and Video Retrieval

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs

Abstract

Dense retrievers powered by pretrained embeddings are widely used for document retrieval but struggle in specialized domains due to the mismatches between the training and target domain distributions. Domain adaptation typically requires costly annotation and retraining of query-document pairs. In this work, we revisit an overlooked alternative: applying PCA to domain embeddings to derive lower-dimensional representations that preserve domain-relevant features while discarding non-discriminative components. Though traditionally used for efficiency, we demonstrate that this simple embedding compression can effectively improve retrieval performance. Evaluated across 9 retrievers and 14 MTEB datasets, PCA applied solely to query embeddings improves NDCG@10 in 75.4% of model-dataset pairs, offering a simple and lightweight method for domain adaptation.

Problem

• domain adaptation
• dense retrieval
• embedding mismatch
• specialized domains
• retrieval performance

Innovation

• embedding compression
• domain adaptation
• dense retrieval
• PCA
• query embedding