Beyond the Sentence: A Survey on Context-Aware Machine Translation with Large Language Models

2025-06-09

AI Summary

Context-aware machine translation (CAMT) remains challenging for large language models (LLMs), particularly in effectively modeling discourse-level context. Method: This work systematically surveys four LLM-based CAMT paradigms—prompt engineering, supervised fine-tuning, automatic post-editing, and translation agents—and introduces dynamic context modeling and multi-turn dialogue-based translation agents as novel approaches. Contribution/Results: It establishes the first comprehensive research taxonomy for LLM-driven CAMT, empirically demonstrating that commercial models (e.g., ChatGPT) significantly outperform mainstream open-source models (e.g., Llama, BLOOM) in contextual understanding. Prompt-based methods are validated as an efficient and competitive baseline. The study provides a unified theoretical framework, an empirical benchmark, and concrete technical directions for advancing CAMT, bridging gaps between LLM capabilities and translation-specific contextual reasoning requirements.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Despite the popularity of the large language models (LLMs), their application to machine translation is relatively underexplored, especially in context-aware settings. This work presents a literature review of context-aware translation with LLMs. The existing works utilise prompting and fine-tuning approaches, with few focusing on automatic post-editing and creating translation agents for context-aware machine translation. We observed that the commercial LLMs (such as ChatGPT and Tower LLM) achieved better results than the open-source LLMs (such as Llama and Bloom LLMs), and prompt-based approaches serve as good baselines to assess the quality of translations. Finally, we present some interesting future directions to explore.

Problem

• Exploring context-aware machine translation using large language models
• Comparing commercial and open-source LLMs for translation quality
• Investigating prompting and fine-tuning approaches for context-aware translation

Innovation

• Utilizing prompting and fine-tuning approaches
• Exploring automatic post-editing techniques
• Developing translation agents for context-awareness