Parameter-Efficient Retrievers for Polish and European Languages

2026-09-11

AI Summary

本文提出了一种三阶段训练流程，通过跨语言对齐、关系知识蒸馏和对比微调来开发紧凑高效的检索器，以解决大规模索引、频繁语料更新及低延迟服务成本高的问题。

Abstract

Dense retrieval systems increasingly rely on multi-billion-parameter language models, whose memory and computational requirements make large-scale indexing, frequent corpus updates, and low-latency serving costly. We present a three-stage training pipeline for developing compact and efficient retrievers that remain competitive with substantially larger models. The pipeline combines cross-lingual alignment, relational knowledge distillation, and contrastive fine-tuning. It requires no original ground-truth relevance labels, relying exclusively on supervision generated by strong embedding models and rerankers utilised as teachers. Using this pipeline, we develop PolDense and EuroDense, both supporting contexts of up to 8,192 tokens. PolDense is a family of six Polish retrievers ranging from 17M to 1B parameters. EuroDense is a 435M-parameter retriever supporting nine European languages. We conduct an extensive evaluation covering 41 Polish and 150 multilingual retrieval tasks. The results demonstrate strong quality-efficiency trade-offs. PolDense-1B outperforms the evaluated retrievers with up to 9B parameters, while the PolDense family forms the Pareto frontier across model sizes. Among the evaluated models below 1B parameters, EuroDense ranks first in both task-averaged and language-averaged performance and leads in seven of nine languages. We release all models publicly.

Problem

• Dense Retrieval
• Parameter Efficiency
• Multilingual Models
• Computational Requirements

Innovation

• Cross-lingual Alignment
• Relational Knowledge Distillation
• Contrastive Fine-tuning