LocQE: Principled Domain Adaptation for Localisation Quality Estimation by Leveraging Post-Edits

2026-09-16

AI Summary

本文针对机器翻译质量评估模型在本地化场景中的不适应问题，通过少量后编辑数据的多任务微调方法，提高了模型对本地化翻译质量的评估能力。

Abstract

Learned quality estimation (QE) models such as COMETKiwi are widespread and work well for general machine translation evaluation. However, they are known to struggle on unseen domains, limiting their performance in a real-world localisation context. We show that they are insensitive to some important factors in localisation, such as whether numbers are translated accurately, or even whether the correct number of spaces and punctuation are preserved in a translation. Further, a key capability for optimisation of machine translation is the ability of QE models to accurately rank different translations of a single segment, which suffers significantly from the domain transfer. In the absence of large-scale direct assessment data, we propose principled fine-tuning approaches to reduce the domain gap with even small amounts of post-editing data. Using a multi-task fine-tuning approach and a simple tokeniser intervention, we create a QE model which proves markedly better at distinguishing preferred post-edits from rejected initial translations in a localisation context. We show that preferences and artificial continuous scores stabilise each other, and argue that to calibrate metrics both in terms of their absolute scores and comparisons between translation of the same source, both types of signal are needed.

Problem

• quality estimation
• domain adaptation
• localisation
• machine translation
• post-edits

Innovation

• Domain Adaptation
• Post-Edits
• Multi-task Fine-tuning