When Life Gives You Samples: The Benefits of Scaling up Inference Compute for Multilingual LLMs

2025-06-25

AI Summary

This paper addresses the weak computational scaling and poor cross-lingual generalization—particularly for low-resource languages—of large language models (LLMs) in multilingual, multitask open-ended generation. We propose a language- and task-aware adaptive inference-time computation scaling method. Our core innovations include: (1) a dynamic temperature–based multilingual sampling strategy; (2) a cross-lingual output selection mechanism grounded in semantic consistency; and (3) a lightweight parallel generation and scoring framework. On the m-ArenaHard-v2.0 benchmark, our approach boosts average win rates by 6.8 percentage points for an 8B model and achieves a 9.0-point gain for an 111B model using only five samples—substantially outperforming single-decoding baselines and prior methods. To our knowledge, this is the first work to enable efficient and inclusive inference-time computation scaling in multilingual settings.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Generation
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Recent advancements in large language models (LLMs) have shifted focus toward scaling inference-time compute, improving performance without retraining the model. A common approach is to sample multiple outputs in parallel, and select one of these as the final output. However, work to date has focused on English and a handful of domains such as math and code. In contrast, we are most interested in techniques that generalize across open-ended tasks, formally verifiable tasks, and across languages. In this work, we study how to robustly scale inference-time compute for open-ended generative tasks in a multilingual, multi-task setting. Our findings show that both sampling strategy based on temperature variation and selection strategy must be adapted to account for diverse domains and varied language settings. We evaluate existing selection methods, revealing that strategies effective in English often fail to generalize across languages. We propose novel sampling and selection strategies specifically adapted for multilingual and multi-task inference scenarios, and show they yield notable gains across languages and tasks. In particular, our combined sampling and selection methods lead to an average +6.8 jump in win-rates for our 8B models on m-ArenaHard-v2.0 prompts, against proprietary models such as Gemini. At larger scale, Command-A (111B model) equipped with our methods, shows +9.0 improvement in win-rates on the same benchmark with just five samples against single-sample decoding, a substantial increase at minimal cost. Our results underscore the need for language- and task-aware approaches to inference-time compute, aiming to democratize performance improvements in underrepresented languages.

Problem

• Scaling inference-time compute for multilingual LLMs
• Adapting sampling strategies for diverse domains and languages
• Improving performance in underrepresented languages and tasks

Innovation

• Scaling inference-time compute for multilingual LLMs
• Adapting sampling strategies for diverse languages
• Novel multilingual-aware selection methods