Improving Term Evaluation in Machine Translation: Variation Matters

2026-09-08

AI Summary

研究通过引入跨术语变异度量，解决机器翻译中术语评估对变体不敏感的问题，改进了英法科学翻译的文档级评估方法。

Abstract

Terminology evaluation in machine translation (MT) usually assumes a single correct target form per source term. However, human translators routinely introduce variation that current metrics penalize as inconsistency. We examine how to account for this variation in document-level MT evaluation of English-French scientific translation, combining glossary-based accuracy, translation consistency, and a new cross-term variation (CTV) diagnostic measure that tests whether variation relationships are preserved across languages. Based on analyses of two parallel corpora, translated by four MT systems, we find that (1) MT systems generate less target-side variation than human translators; (2) transfer patterns strongly depend on the variation type; (3) consistency rankings vary with the choice of metric; and (4) constraining MT with a glossary improves accuracy and consistency but degrades CTV by suppressing valid variation. We argue for variation-aware evaluation that conditions consistency penalties on whether target-side variation mirrors source-side variation.

Problem

• Term Evaluation
• Machine Translation
• Variation
• Consistency

Innovation

• cross-term variation (CTV)
• variation-aware evaluation
• glossary-based accuracy