Using Source-Side Confidence Estimation for Reliable Translation into Unfamiliar Languages

2025-03-30

AI Summary

To address the insufficient reliability and interpretability of machine translation for low-resource languages, this paper proposes a novel source-side confidence estimation paradigm that does not require word alignment. It directly quantifies the sensitivity of target token probabilities to infinitesimal perturbations in source embeddings via gradient-based sensitivity analysis, thereby identifying potential mistranslations. Building upon this, we design an interactive human-in-the-loop translation framework enabling real-time user intervention—even by individuals with limited target-language proficiency. Furthermore, integrating neural machine translation (NMT) model fine-tuning with interpretability-enhancing techniques, our approach significantly outperforms alignment-based baselines across multilingual low-resource settings: mistranslation detection F1 improves by 12.7%, user intervention efficiency increases by 34%, and cross-lingual communication gains enhanced trustworthiness and transparency.

Technology Category

• Natural Language Processing: Machine Translation, Multilinguality, Cross-Lingual NLP
• Machine Learning: Transparent, Interpretable, Explainable ML
• Humans and AI: Human-in-the-loop Machine Learning

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

We present an interactive machine translation (MT) system designed for users who are not proficient in the target language. It aims to improve trustworthiness and explainability by identifying potentially mistranslated words and allowing the user to intervene to correct mistranslations. However, confidence estimation in machine translation has traditionally focused on the target side. Whereas the conventional approach to source-side confidence estimation would have been to project target word probabilities to the source side via word alignments, we propose a direct, alignment-free approach that measures how sensitive the target word probabilities are to changes in the source embeddings. Experimental results show that our method outperforms traditional alignment-based methods at detection of mistranslations.

Problem

• Improving trustworthiness in machine translation for unfamiliar languages
• Detecting mistranslations via source-side confidence estimation
• Proposing alignment-free method for better mistranslation detection

Innovation

• Direct source-side confidence estimation without alignments
• Measures sensitivity of target probabilities to source changes
• Outperforms traditional alignment-based mistranslation detection