TranslateGemma Technical Report

2026-01-13

AI Summary

This work proposes a two-stage fine-tuning strategy to enhance the performance and efficiency of open-source large language models in multilingual machine translation, building upon the Gemma 3 base model. The approach first applies supervised fine-tuning using high-quality synthetic and human-curated parallel corpora, followed by a novel integration of reinforcement learning with multiple reward models—MetricX-QE and AutoMQM—applied for the first time to the Gemma series to optimize translation quality. While preserving multimodal capabilities, the method substantially improves translation performance even in smaller models. Experimental results demonstrate consistent and significant gains over baselines on both WMT25 human evaluation (across 10 language pairs) and WMT24++ automatic evaluation (across 55 language pairs), with the smaller models often matching or surpassing the performance of larger counterparts, and showing enhanced results on the Vistra image-to-text translation benchmark.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Machine Translation, Multilinguality, Cross-Lingual NLP
• Computer Vision: Large Vision Models

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

We present TranslateGemma, a suite of open machine translation models based on the Gemma 3 foundation models. To enhance the inherent multilingual capabilities of Gemma 3 for the translation task, we employ a two-stage fine-tuning process. First, supervised fine-tuning is performed using a rich mixture of high-quality large-scale synthetic parallel data generated via state-of-the-art models and human-translated parallel data. This is followed by a reinforcement learning phase, where we optimize translation quality using an ensemble of reward models, including MetricX-QE and AutoMQM, targeting translation quality. We demonstrate the effectiveness of TranslateGemma with human evaluation on the WMT25 test set across 10 language pairs and with automatic evaluation on the WMT24++ benchmark across 55 language pairs. Automatic metrics show consistent and substantial gains over the baseline Gemma 3 models across all sizes. Notably, smaller TranslateGemma models often achieve performance comparable to larger baseline models, offering improved efficiency. We also show that TranslateGemma models retain strong multimodal capabilities, with enhanced performance on the Vistra image translation benchmark. The release of the open TranslateGemma models aims to provide the research community with powerful and adaptable tools for machine translation.

Problem

• machine translation
• multilingual
• translation quality
• foundation models
• open models

Innovation

• TranslateGemma
• two-stage fine-tuning
• reinforcement learning for translation
• multilingual machine translation
• open-source MT models