BatCoder: Self-Supervised Bidirectional Code-Documentation Learning via Back-Translation

2026-01-30

AI Summary

This work addresses the scarcity of high-quality code-documentation pairs in niche programming languages, which limits the performance of large code models. The authors propose a self-supervised reinforcement learning framework that jointly optimizes code and documentation generation through a back-translation mechanism: documentation is first generated from code, then new code is reconstructed from that documentation, with the semantic similarity between the original and reconstructed code serving as an implicit reward signal. Requiring only raw code data and no human annotations, this approach substantially expands the scale of effective training data. Evaluated on a 7B-parameter model, the method achieves pass@1 scores of 83.5% on HumanEval and 81.0% on MBPP, outperforming mainstream open-source baselines, with consistent performance gains observed as both data volume and model size increase.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Training LLMs for code-related tasks typically depends on high-quality code-documentation pairs, which are costly to curate and often scarce for niche programming languages. We introduce BatCoder, a self-supervised reinforcement learning framework designed to jointly optimize code generation and documentation production. BatCoder employs a back-translation strategy: a documentation is first generated from code, and then the generated documentation is used to reconstruct the original code. The semantic similarity between the original and reconstructed code serves as an implicit reward, enabling reinforcement learning to improve the model's performance both in generating code from documentation and vice versa. This approach allows models to be trained using only code, substantially increasing the available training examples. Evaluated on HumanEval and MBPP with a 7B model, BatCoder achieved 83.5% and 81.0% pass@1, outperforming strong open-source baselines. Moreover, the framework demonstrates consistent scaling with respect to both training corpus size and model capacity.

Problem

• code-documentation pairs
• data scarcity
• programming languages
• LLM training

Innovation

• self-supervised learning
• back-translation
• code-documentation alignment
• reinforcement learning
• code generation