Please Translate Again: Two Simple Experiments on Whether Human-Like Reasoning Helps Translation

2025-06-05

AI Summary

It remains unclear whether Chain-of-Thought (CoT) prompting genuinely enhances translation performance in large language models (LLMs), particularly given the prevailing assumption that explicit task decomposition improves quality. Method: We conduct zero-shot comparative experiments on the WMT24 benchmark, systematically evaluating handcrafted multi-step reasoning prompts against a minimalist retry prompt (“Please translate again”). Contribution/Results: The retry prompt significantly outperforms state-of-the-art CoT translation methods, challenging the hypothesis that explicit reasoning structures inherently boost translation accuracy. Our analysis indicates that observed gains stem not from logical decomposition per se, but rather from increased output diversity and resampling effects—i.e., stochastic re-generation yielding higher-probability, more fluent translations. This finding reframes LLM translation prompting: simplicity and pragmatic efficacy should supersede artificial emulation of human-like reasoning. The work provides empirical grounding for minimalist prompt design and calls for re-evaluating the role of structured reasoning in machine translation with LLMs.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Prompt Engineering / Prompting
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Large Language Models (LLMs) demonstrate strong reasoning capabilities for many tasks, often by explicitly decomposing the task via Chain-of-Thought (CoT) reasoning. Recent work on LLM-based translation designs hand-crafted prompts to decompose translation, or trains models to incorporate intermediate steps.~ extit{Translating Step-by-step}~citep{briakou2024translating}, for instance, introduces a multi-step prompt with decomposition and refinement of translation with LLMs, which achieved state-of-the-art results on WMT24. In this work, we scrutinise this strategy's effectiveness. Empirically, we find no clear evidence that performance gains stem from explicitly decomposing the translation process, at least for the models on test; and we show that simply prompting LLMs to ``translate again'' yields even better results than human-like step-by-step prompting. Our analysis does not rule out the role of reasoning, but instead invites future work exploring the factors for CoT's effectiveness in the context of translation.

Problem

• Does human-like reasoning improve LLM translation performance
• Evaluating effectiveness of step-by-step translation decomposition
• Comparing 'translate again' prompt vs. structured reasoning prompts

Innovation

• Using Chain-of-Thought reasoning for translation
• Multi-step prompt with decomposition and refinement
• Simple 'translate again' prompt outperforms step-by-step