LLaMAX2: Your Translation-Enhanced Model also Performs Well in Reasoning

2025-10-10

AI Summary

General-purpose large language models (LLMs) exhibit strong reasoning capabilities but weak translation performance, whereas translation-specialized models often sacrifice reasoning ability. Method: We propose a translation-augmentation paradigm based on instruction tuning, incorporating a layer-selective fine-tuning strategy that leverages only small-scale multilingual parallel data to jointly optimize translation and reasoning capabilities. Contribution/Results: Our approach is the first to significantly improve low-resource language translation—e.g., +15+ spBLEU and +40+ xCOMET for Swahili—without degrading original reasoning performance. It achieves an average gain of over 1.0 point across seven multilingual tasks, while maintaining baseline-level performance on 15 mainstream reasoning benchmarks. The core innovation lies in decoupling translation enhancement from reasoning degradation, thereby substantially reducing the complexity of multilingual adaptation.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Knowledge Representation and Reasoning: Argumentation

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

General Large Language Models (LLMs) excel in reasoning, but those enhanced for translation struggle with reasoning tasks. To address this, we propose a novel translationenhanced recipe that begins with instruct models and applies layer-selective tuning only on parallel data. Following this pipeline, we introduce the Qwen3-XPlus models, which demonstrate significant improvements in translation performance across both high- and lowresource languages, achieving 15+ spBLEU and 40+ xComet in low-resource languages, like Swahili. Interestingly, training only with small parallel datasets, Qwen3-XPlus achieves an average improvement of 1+ points on 7 multilingual tasks while maintaining proficiency comparable to the Qwen3 instruct model in 15 popular reasoning datasets. This work offers a promising approach to multilingual enhancement, significantly reducing complexity and enhancing accessibility for a wider range of languages. The code and model are publicly available.

Problem

• Addressing reasoning performance decline in translation-enhanced large language models
• Developing multilingual models with improved translation and preserved reasoning capabilities
• Reducing complexity while enhancing performance across high- and low-resource languages

Innovation

• Layer-selective tuning on parallel data
• Translation-enhanced recipe from instruct models
• Multilingual enhancement with small parallel datasets