Domain-Specific Quality Estimation for Machine Translation in Low-Resource Scenarios

2026-03-07

AI Summary

This work addresses the challenge of sentence-level quality estimation (QE) for English-to-Indic language translation in low-resource, reference-free scenarios, particularly within domain-specific contexts such as healthcare and legal texts. The authors propose integrating Low-Rank Adaptation (LoRA) into intermediate layers of large language models, coupled with a regression head, and further extend this approach to Low-Rank Multiplicative Adaptation (LoRMA). Their method demonstrates significantly improved QE stability across zero-shot, few-shot, and instruction-prompting settings, with especially strong performance on semantically complex domain-specific content. Experiments conducted under the ALOPE framework validate the effectiveness of the approach on both open-source and closed-source models. The authors also release their code and a new domain-specific QE dataset to support future research.

Technology Category

• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Machine Learning: Large Multimodal Models (LMMs)
• Knowledge Representation and Reasoning: Qualitative Reasoning

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

Quality Estimation (QE) is essential for assessing machine translation quality in reference-less settings, particularly for domain-specific and low-resource language scenarios. In this paper, we investigate sentence-level QE for English to Indic machine translation across four domains (Healthcare, Legal, Tourism, and General) and five language pairs. We systematically compare zero-shot, few-shot, and guideline-anchored prompting across selected closed-weight and open-weight LLMs. Findings indicate that while closed-weight models achieve strong performance via prompting alone, prompt-only approaches remain fragile for open-weight models, especially in high-risk domains. To address this, we adopt ALOPE, a framework for LLM-based QE that uses Low-Rank Adaptation with regression heads attached to selected intermediate Transformer layers. We also extend ALOPE with recently proposed Low-Rank Multiplicative Adaptation (LoRMA). Our results show that intermediate-layer adaptation consistently improves QE performance, with gains in semantically complex domains, indicating a path toward more robust QE in practical scenarios. We release code and domain-specific QE datasets publicly to support further research.

Problem

• Quality Estimation
• Machine Translation
• Low-Resource
• Domain-Specific
• Reference-less

Innovation

• Quality Estimation
• Low-Rank Adaptation
• Domain-Specific MT
• LLM Prompting
• LoRMA