Decoding Machine Translationese in English-Chinese News: LLMs vs. NMTs

2025-06-27

AI Summary

This study systematically investigates Machine Translationese (MTese) in English-to-Chinese news translation, offering the first comparative analysis of MTese characteristics between Neural Machine Translation (NMT) systems and Large Language Models (LLMs) within a Chinese linguistic context. We construct a large-scale, bilingual-aligned corpus comprising four sub-corpora and propose a five-layer linguistic feature framework—covering syntax, lexis, conjunctions, punctuation, and lexical diversity—analyzed via chi-square tests, supervised classification, and unsupervised clustering. Results show native Chinese and machine-translated texts are nearly perfectly distinguishable (>95% classification accuracy); NMT exhibits preferences for shorter sentences, high-frequency adversative conjunctions, and parenthetical punctuation, whereas LLMs demonstrate greater lexical richness; no significant differences emerge between domestic and international LLMs; and NMT–LLM discrimination achieves 70% accuracy. This work establishes a novel empirical foundation and methodological framework for MTese detection, translation quality assessment, and understanding of LLM translation behavior.

Technology Category

• Natural Language Processing: Machine Translation, Multilinguality, Cross-Lingual NLP
• Machine Learning: Large Multimodal Models (LMMs)
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search

Abstract

This study explores Machine Translationese (MTese) -- the linguistic peculiarities of machine translation outputs -- focusing on the under-researched English-to-Chinese language pair in news texts. We construct a large dataset consisting of 4 sub-corpora and employ a comprehensive five-layer feature set. Then, a chi-square ranking algorithm is applied for feature selection in both classification and clustering tasks. Our findings confirm the presence of MTese in both Neural Machine Translation systems (NMTs) and Large Language Models (LLMs). Original Chinese texts are nearly perfectly distinguishable from both LLM and NMT outputs. Notable linguistic patterns in MT outputs are shorter sentence lengths and increased use of adversative conjunctions. Comparing LLMs and NMTs, we achieve approximately 70% classification accuracy, with LLMs exhibiting greater lexical diversity and NMTs using more brackets. Additionally, translation-specific LLMs show lower lexical diversity but higher usage of causal conjunctions compared to generic LLMs. Lastly, we find no significant differences between LLMs developed by Chinese firms and their foreign counterparts.

Problem

• Identify Machine Translationese in English-Chinese news texts
• Compare linguistic patterns between NMTs and LLMs outputs
• Analyze feature differences in translation-specific vs generic LLMs

Innovation

• Large dataset with five-layer feature set
• Chi-square ranking for feature selection
• Compare LLMs and NMTs using classification