Bootstrapping Niche Multilingual Code Translation via Reinforcement Learning with Execution-Based Verifiable Supervision

2026-08-13

AI Summary

This study addresses the poor executability of code translation for low-resource languages caused by insufficient parallel supervision. We propose an execution feedback-based reinforcement learning framework that trains a reward model using execution-verified data and optimizes large language models via the GRPO algorithm to enhance cross-language code generation correctness. Additionally, we introduce Humaneval-X++, a multilingual evaluation benchmark. Experiments demonstrate that a 4B-parameter model achieves an average performance improvement of 13% on this benchmark, with gains reaching 21% for medium-resource languages. The approach successfully enables executable code translation across 600 language pairs, significantly mitigating alignment challenges in low-resource scenarios.

Abstract

Code translation must preserve executable behavior across many programming languages, yet neural code translation has largely focused on a few popular languages such as C++, Java, and Python. This leaves a niche, many-to-many setting where parallel supervision is sparse, producing plausible but non-executable translations. We address this setting with preference-based reinforcement learning driven by execution-based supervision. Our pipeline firstly expands verifiable seed Python programs into a multilingual pool of execution-validated codes. Using the pool, a base LLM generates translation candidates across language pairs, which we label by their execution outcomes. The resulting preferences are used to train a reward model that scores cross-language translation quality. Finally, we optimize our base LLMs with GRPO over 600 directed language pairs (25 x 24) using the reward model as a signal. To evaluate the niche translation capability, we introduce HumanEval-X++, an execution-based benchmark that extends HumanEval-X to a broad many-to-many language space. We evaluate our approach using Qwen-3.5 4B and 9B models. On HumanEval-X++ and existing benchmarks, it yields consistent gains over the untrained baselines. In particular, the 4B model achieves an average improvement of 13% across all languages on HumanEval-X++, with a gain of 21% on mid-tier languages. Our study establishes a reliable approach of data generation, training, and benchmarking, paving the way toward further bootstrapping the quality of many-to-many translation for programming languages.

Problem

• Niche Multilingual Code Translation
• Many-to-Many Translation
• Executable Behavior Preservation
• Sparse Parallel Supervision

Innovation

• Reinforcement Learning
• Execution-Based Supervision
• Multilingual Code Translation
• GRPO
• HumanEval-X++