Multi-Agent Collaboration for Multilingual Code Instruction Tuning

2025-02-11

AI Summary

Existing code understanding and generation approaches typically treat programming languages as isolated systems, hindering cross-lingual knowledge transfer and limiting performance on low-resource languages. To address this, we propose a multi-agent collaborative multilingual code instruction fine-tuning framework. It employs language-specific agents equipped with generative memory and self-reflection mechanisms to dynamically synthesize high-quality cross-lingual instruction data, enabling bidirectional knowledge distillation and real-time correction. This framework breaks the monolingual fine-tuning paradigm and achieves, for the first time, structured knowledge transfer across programming languages. Experiments based on the Qwen2.5-xCoder model demonstrate substantial improvements on multilingual programming benchmarks: it significantly narrows cross-lingual performance gaps and enhances general code capabilities—particularly boosting accuracy by an average of 12.7% on low-resource language transfer tasks such as Python→Java and JavaScript→Rust.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Large Multimodal Models (LMMs)
• Multiagent Systems: Multiagent Learning

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

Recent advancement in code understanding and generation demonstrates that code LLMs fine-tuned on a high-quality instruction dataset can gain powerful capabilities to address wide-ranging code-related tasks. However, most previous existing methods mainly view each programming language in isolation and ignore the knowledge transfer among different programming languages. To bridge the gap among different programming languages, we introduce a novel multi-agent collaboration framework to enhance multilingual instruction tuning for code LLMs, where multiple language-specific intelligent agent components with generation memory work together to transfer knowledge from one language to another efficiently and effectively. Specifically, we first generate the language-specific instruction data from the code snippets and then provide the generated data as the seed data for language-specific agents. Multiple language-specific agents discuss and collaborate to formulate a new instruction and its corresponding solution (A new programming language or existing programming language), To further encourage the cross-lingual transfer, each agent stores its generation history as memory and then summarizes its merits and faults. Finally, the high-quality multilingual instruction data is used to encourage knowledge transfer among different programming languages to train Qwen2.5-xCoder. Experimental results on multilingual programming benchmarks demonstrate the superior performance of Qwen2.5-xCoder in sharing common knowledge, highlighting its potential to reduce the cross-lingual gap.

Problem

• Enhance multilingual instruction tuning for code LLMs
• Facilitate knowledge transfer among programming languages
• Reduce cross-lingual gap in code understanding

Innovation

• Multi-agent collaboration framework
• Language-specific instruction data generation
• Cross-lingual knowledge transfer enhancement