CAT-Translate: Building Compact Open-Source Models for Japanese-English Translation

2026-06-19

AI Summary

This study investigates whether specialized lightweight models outperform general-purpose multilingual large language models in Japanese–English translation scenarios. We develop dedicated translation models ranging from 0.8B to 7B parameters, employing a two-stage supervised fine-tuning pipeline followed by multi-objective GRPO reinforcement learning, and further adapt them to target domains using synthetically generated parallel corpora. Experimental results demonstrate that our approach achieves state-of-the-art performance on WMT benchmarks and significantly surpasses leading multilingual models across real-world business domains—including commerce, legal, medical, financial, and patent translation—thereby validating the practical efficacy of lightweight, domain-specialized machine translation systems and advancing the associated open-source ecosystem.

Abstract

Nowadays, large multilingual translation models demonstrate impressive translation capabilities in the machine translation benchmarks. This raises a practical question to the developers: is it worth developing translation models specialized for a particular language pair if you only need to support that language pair? To give an anecdotal answer to this question, we develop a family of small language models (0.8B, 1.4B, 3.3B, and 7B parameters) specialized for Japanese-English bidirectional translation. We employ a two-stage supervised fine-tuning approach followed by Multi-Objective GRPO (Ichihara et al. 2025) to train models on synthetically generated parallel corpora. We evaluate our models on WMT and real-world translation benchmarks across business, legal, medical, financial, and patent domains. While multilingual models achieve strong performance on WMT benchmarks, our compact models outperform them on real-world benchmarks, suggesting the practical utility of developing specialized translation models even in the era of large multilingual models.

Problem

• machine translation
• language pair specialization
• multilingual models
• Japanese-English translation
• compact models

Innovation

• specialized translation models
• synthetic parallel corpora
• Multi-Objective GRPO
• compact language models
• Japanese-English translation