Seed-X: Building Strong Multilingual Translation LLM with 7B Parameters

2025-07-17

AI Summary

To address challenges in multilingual translation—including complex language patterns and unnatural outputs—this paper proposes an optimization framework for 7B-parameter large language models. Methodologically, it integrates high-quality monolingual/bilingual mixed pretraining, chain-of-thought-guided instruction tuning, and reinforcement learning–based optimization of translation generalization across 28 languages. Crucially, it explicitly models reasoning steps within translation instructions and enhances cross-lingual consistency via a multilingual-aligned reward mechanism. Experimental results demonstrate significant improvements over comparable open-source models (e.g., Qwen2-7B, LLaMA3-8B) in both automatic metrics (BLEU, COMET) and human evaluations (fluency, faithfulness), achieving performance on par with Gemini-2.5 and GPT-4o. The model and training paradigm are fully open-sourced, establishing a new benchmark and practical pathway for efficient multilingual translation research.

Technology Category

• Natural Language Processing: Machine Translation, Multilinguality, Cross-Lingual NLP
• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Multilingual translation stands as a challenging task for large language models (LLMs) to handle intricate language patterns and stilted translations that arise in automated translations. In this paper, we introduce Seed-X, a family of open-source LLMs comprising instruct and reasoning models, pushing the limits of translation capability with 7B parameter size. The base model is pre-trained on a diverse, high-quality dataset encompassing both monolingual and bilingual content across 28 languages, harnessing the full potential of multilingual data. The instruct model is then finetuned to translate by Chain-of-Thought (CoT) reasoning and further enhanced through reinforcement learning (RL) to achieve better generalization across diverse language pairs. Seed-X achieves performance comparable to leading closed-source models, including Gemini-2.5 and GPT-4o, across 28 languages, and significantly outperforms larger open-source models in both automatic metrics and human evaluations. We share the best practices through our optimization process, and make the parameter public available for advancing translation research and applications.

Problem

• Addressing intricate language patterns in multilingual translation
• Overcoming stilted translations in automated language models
• Enhancing translation quality across 28 diverse languages

Innovation

• 7B parameter multilingual LLM for translation
• Chain-of-Thought reasoning for instruct model
• Reinforcement learning enhances generalization