Reference-Free Post-Training of Open Large Language Models for Multilingual Machine Translation

2026-08-11

AI Summary

This study addresses the challenge of improving multilingual translation quality for open-source large language models in the absence of reference translations. The authors propose MiLMMT-46-v1.0, a novel approach that integrates reference-free quality estimation and language identification gating into Group Relative Policy Optimization (GRPO) for reinforcement learning-based post-training. Additionally, the method employs linear interpolation to combine supervised fine-tuning (SFT) and reinforcement learning (RL) checkpoints. Experimental results demonstrate that this technique substantially outperforms the original SFT model and established open-source baselines such as Seed-X, HY-MT2, and TranslateGemma. Remarkably, it even surpasses proprietary systems—including Google Translate, Gemini 3 Pro, and GPT-5—across 46 languages, thereby validating the efficacy of checkpoint interpolation over on-policy distillation.

Abstract

We study reference-free post-training for multilingual machine translation with open large language models. Starting from the supervised-finetuned MiLMMT-46-v0.1 models, we apply Group Relative Policy Optimization (GRPO) with a reward that averages two reference-free quality estimation models and is gated by language identification. We then linearly interpolate the supervised fine-tuning (SFT) and reinforcement learning (RL) model checkpoints to obtain MiLMMT-46-v1.0. Across 46 languages, the resulting models consistently improve translation quality over their SFT counterparts, outperform strong recent open baselines, including Seed-X, HY-MT2, and TranslateGemma, and achieve leading reference-free scores against evaluated proprietary systems such as Google Translate, Gemini 3 Pro, and GPT-5. We further investigate on-policy distillation and find that it reaches, but does not surpass, the quality frontier achieved by RL with checkpoint interpolation. We release the models and code to facilitate future research.

Problem

• reference-free
• post-training
• multilingual machine translation
• large language models

Innovation

• reference-free
• Group Relative Policy Optimization
• multilingual machine translation
• checkpoint interpolation
• quality estimation