Think Multilingual, Not Harder: A Data-Efficient Framework for Teaching Reasoning Models to Code-Switch

2026-04-16

AI Summary

This work addresses the limited capability of current reasoning models to effectively handle code-switching—the practice of alternating between multiple languages within a single utterance—which is often misclassified as erroneous or processed only under restrictive conditions. To overcome this, the study introduces the first fine-tuning framework grounded in linguistic theory and behavioral motivation. By constructing a multilingual, multitask dataset of reasoning trajectories, the authors analyze beneficial instances of code-switching and devise a data-efficient fine-tuning strategy. The approach integrates behavior-guided fine-tuning interventions with cross-task transfer—particularly leveraging translation tasks—to substantially enhance the model’s ability to generate advantageous code-switching during reasoning. Notably, this improvement is achieved with minimal training data, demonstrating both high efficiency and strong generalization potential.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Large Multimodal Models (LMMs)
• Knowledge Representation and Reasoning: Reasoning with Beliefs

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Recent developments in reasoning capabilities have enabled large language models to solve increasingly complex mathematical, symbolic, and logical tasks. Interestingly, while reasoning models are often trained to generate monolingual text, these models have also been observed to code-switch (i.e., mix languages). Prior works have either viewed code-switching as an undesirable error, attempted to control code-switching through modifications to input prompts or the output decoding process, or focus on narrow subsets of languages, domains, tasks, and models. We address these gaps by introducing the first linguistically and behaviorally motivated fine-tuning framework for identifying beneficial code-switched reasoning behaviors in large language models and teaching these models to code-switch more effectively for reasoning. First, we create and systematically analyze a dataset of reasoning traces from diverse models, languages, tasks, and domains to understand the types of code-switching behaviors found in existing reasoning models. Then, we develop fine-tuning interventions that teach reasoning models to code-switch based on our observations of helpful behaviors in existing models. We find that our framework can significantly increase beneficial code-switched reasoning behaviors in a data-efficient manner. Interestingly, we also find that code-switching behaviors in reasoning models can be modified by fine-tuning for tasks that do not directly demonstrate code-switching in reasoning (e.g., machine translation). Our work suggests that data-efficient interventions can instill helpful forms of code-switching behavior in reasoning models.

Problem

• code-switching
• reasoning models
• multilingual
• large language models
• data-efficient

Innovation

• code-switching
• reasoning models
• data-efficient fine-tuning
• multilingual reasoning
• behavioral intervention