Discourse Dependency: A Continuous Criterion for Translation Difficulty

2026-09-04

AI Summary

本文提出了一种新的翻译难度度量方法——话语依赖（DDP），并通过实验证明了在高DDP段落中，当前的机器翻译系统难以匹敌人工翻译。

Abstract

Recent calls for harder machine translation benchmarks have not clarified what difficulty should mean. We argue that one meaningful and currently unmeasured axis is referential reach, the distance a segment must look back into its document to resolve the entities and pronouns it contains. We formalize this as discourse dependency (DDP), a metric-free, source-side measure computed from named entity re-mentions and pronominal coreference. Validated against gold coreference, DDP errs one-sidedly in 99.2% of segments, so a high-DDP segment is certified to require long-range context. Applying DDP to WMT24++ and WMT25 shows that both are heavily skewed toward low-DDP segments, which domain labels do not distinguish. Building on DDP, we compare five context injection strategies in an English-Korean post-editing setup, varying context size and selection. As DDP grows, no strategy keeps pace with human post-editing. On segments with DDP >= 15 raters prefer human translations, while automatic metrics register no difference. As frontier systems saturate aggregate scores, DDP shifts evaluation from how well models score to how far they can reach.

Problem

• Translation Difficulty
• Referential Reach
• Discourse Dependency

Innovation

• discourse dependency
• referential reach
• context injection strategies
• translation difficulty