Is continuous CoT better suited for multi-lingual reasoning?

2026-03-09

AI Summary

This work investigates the efficacy of Continuous Chain-of-Thought (CoT) reasoning for multilingual inference, particularly in enhancing model robustness under low-resource and zero-shot cross-lingual settings. Building upon the CODI framework, the approach performs reasoning in a continuous latent space and is evaluated across GSM8k and CommonsenseQA benchmarks in multiple languages. Experiments span five typologically diverse languages and demonstrate, for the first time, that continuous implicit reasoning exhibits strong language invariance: it achieves superior performance on unseen languages in zero-shot scenarios, compresses reasoning trajectories by 29–50×, and significantly outperforms conventional explicit CoT and standard supervised fine-tuning methods, thereby offering an efficient and scalable solution for cross-lingual reasoning.

Technology Category

• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Natural Language Processing: Safety and Robustness
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

We investigate whether performing reasoning in a continuous latent space leads to more robust multilingual capabilities. We compare Continuous Chain-of-Thought (using the CODI framework) against standard supervised fine-tuning across five typologically diverse languages: English, Chinese, German, French, and Urdu. Our experiments on GSM8k and CommonsenseQA demonstrate that continuous reasoning significantly outperforms explicit reasoning on low-resource languages, particularly in zero-shot settings where the target language was not seen during training. Additionally, this approach achieves extreme efficiency, compressing reasoning traces by approximately $29\times$ to $50\times$. These findings indicate that continuous latent representations naturally exhibit greater language invariance, offering a scalable solution for cross-lingual reasoning.

Problem

• multilingual reasoning
• low-resource languages
• zero-shot settings
• language invariance
• cross-lingual reasoning

Innovation

• Continuous Chain-of-Thought
• multilingual reasoning
• language invariance
• zero-shot transfer
• reasoning compression