AdaMame: A Training Recipe for Adaptive Multilingual Reasoning

2026-06-12

AI Summary

This work addresses the language collapse problem in large reasoning models under multilingual settings, where models struggle to maintain consistent reasoning in the query language. Existing approaches fail to simultaneously achieve high accuracy, strong language fidelity, and efficient reasoning. To overcome this, the authors propose AdaMame, a two-stage training framework: first, supervised fine-tuning on natural inference trajectories across five languages, followed by reinforcement learning with a novel AdaMame-GRPO algorithm. This algorithm employs a query-conditioned, progressive language alignment mechanism that dynamically adjusts alignment strength to shift the model’s focus from exploration to the query language. AdaMame achieves a Pareto-optimal balance—preserving accuracy while maximizing language fidelity and minimizing token consumption. Experiments demonstrate state-of-the-art performance across two benchmarks, two large reasoning models, and twelve languages, with particularly notable gains for out-of-domain low-resource languages.

Abstract

While Large Reasoning Models (LRMs) show strong performance in English, they often fail to reason in the language of the query, a phenomenon known as language collapse. Existing RL-based fixes typically add a binary language fidelity reward to the accuracy objective, yet still incur trade-off in accuracy, mid-trace code-switching, and excessive token usage. In this work, we propose AdaMame, a two-stage training recipe for multilingual mathematical reasoning that addresses these limitations by adaptively aligning the reasoning language to the query language without compromising accuracy. The first SFT stage fine-tunes on naturally occurring reasoning traces across five languages to establish multilingual reasoning capability. In the subsequent RL stage, we introduce AdaMame-GRPO, an adaptation of Group Relative Policy Optimization (GRPO) in which a query-conditioned alignment factor grows progressively during training, guiding the model to first explore diverse reasoning languages before exploiting reasoning in the query language. Evaluated across two benchmarks, two LRMs, and 12 languages, AdaMame-GRPO achieves Pareto-optimal performance across reasoning accuracy, language fidelity, and token efficiency over all baselines, with the strongest gains on out-of-domain, lower-resource languages.

Problem

• language collapse
• multilingual reasoning
• Large Reasoning Models
• language fidelity
• token efficiency

Innovation

• AdaMame
• multilingual reasoning
• language alignment
• GRPO
• reinforcement learning