Efficient Multilingual Reasoning Transfer via Progressive Code-Switching

2026-07-01

AI Summary

This work addresses the significant performance degradation of large language reasoning models in non-English languages and the high cost of existing transfer approaches that rely on distillation from stronger models or external supervision. The authors propose a Progressive Code-Switching (PCS) framework that requires only lightweight translation. By leveraging supervised fine-tuning to activate the model’s inherent code-mixing capability, combined with reinforcement learning and a curriculum strategy that gradually increases the proportion of the target language, PCS enables smooth transfer from English to the target language. Notably, this approach eliminates the need for stronger teacher models or external evaluators. As the first method to integrate code-switching with curriculum learning for multilingual reasoning, PCS substantially narrows the performance gap with English across five languages and multiple benchmarks while simultaneously improving linguistic consistency and accuracy.

Abstract

Large reasoning models (LRMs) have achieved strong reasoning capabilities in English, yet their performance degrades significantly when required to reason in other languages. A natural solution is to transfer the model's English reasoning ability to target languages. However, existing transfer approaches typically rely on distilled target-language reasoning traces from stronger LRMs or online supervision from external judge models, which are costly and difficult to scale. In this paper, we propose PCS (Progressive Code-Switching), a more efficient transfer framework that requires only lightweight translation without any stronger model for distillation or judging. PCS first constructs code-switched reasoning traces by translating a subset of English reasoning steps into the target language, and uses them to initialize the model's code-switching ability via supervised fine-tuning. It then applies reinforcement learning with a step-level language consistency curriculum, progressively raising the target-language ratio until the model reasons entirely in the target language. This progressive design provides a smooth transfer path that avoids the instability and performance degradation commonly observed when directly enforcing target-language reasoning. Experiments on multiple benchmarks and five typologically diverse languages show that PCS substantially narrows the performance gap between target-language and English reasoning, yielding more language-consistent reasoning while maintaining competitive accuracy.

Problem

• multilingual reasoning
• reasoning transfer
• code-switching
• language gap
• large reasoning models

Innovation

• Progressive Code-Switching
• Multilingual Reasoning Transfer
• Reinforcement Learning
• Language Consistency Curriculum
• Supervised Fine-tuning