Improving Retrieval-Augmented Neural Machine Translation with Monolingual Data

2025-04-30

AI Summary

In low-resource neural machine translation (NMT), parallel bilingual corpora are scarce, yet monolingual target-language data are often abundant. Method: This paper proposes the first cross-lingual retrieval-augmented NMT framework explicitly leveraging monolingual target-language corpora—bypassing conventional bilingual memory banks. It retrieves relevant target-language segments directly using the source sentence as a query and introduces a novel joint sentence- and word-level contrastive learning objective to enforce multi-granularity semantic alignment, seamlessly integrated into the RANMT architecture. Contribution/Results: Experiments demonstrate that our approach significantly outperforms baseline NMT models and generic cross-lingual retrievers in both controlled settings and realistic low-resource scenarios. Notably, when the scale of target monolingual data vastly exceeds that of available parallel data, the method yields substantial BLEU improvements, validating its effectiveness in data-imbalanced low-resource regimes.

Technology Category

• Natural Language Processing: Machine Translation, Multilinguality, Cross-Lingual NLP
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Conventional retrieval-augmented neural machine translation (RANMT) systems leverage bilingual corpora, e.g., translation memories (TMs). Yet, in many settings, in-domain monolingual target-side corpora are often available. This work explores ways to take advantage of such resources by retrieving relevant segments directly in the target language, based on a source-side query. For this, we design improved cross-lingual retrieval systems, trained with both sentence level and word-level matching objectives. In our experiments with two RANMT architectures, we first demonstrate the benefits of such cross-lingual objectives in a controlled setting, obtaining translation performances that surpass standard TM-based models. We then showcase our method on a real-world set-up, where the target monolingual resources far exceed the amount of parallel data and observe large improvements of our new techniques, which outperform both the baseline setting, and general-purpose cross-lingual retrievers.

Problem

• Enhancing retrieval-augmented NMT with monolingual target data
• Improving cross-lingual retrieval using sentence and word-level objectives
• Outperforming TM-based models with large monolingual resources

Innovation

• Utilizes monolingual target-side data for retrieval
• Trains cross-lingual retrievers with multi-level objectives
• Outperforms traditional TM-based and general retrievers