Rethinking Cross-lingual Gaps from a Statistical Viewpoint

2025-10-17

AI Summary

This paper addresses the cross-lingual gap—the degradation in knowledge retrieval accuracy of large language models (LLMs) in target languages—challenging the prevailing assumption that latent representation misalignment is the primary cause. Method: We identify response variance as the dominant factor and introduce the first bias–variance decomposition framework tailored to cross-lingual settings, theoretically establishing variance as the principal driver of performance decline. Building on this insight, we propose variance control—not just representation alignment—as the key mechanism for bridging the gap. We further design lightweight, inference-time interventions (e.g., prompt instruction optimization) that require no model fine-tuning or architectural modification. Results: Our approach significantly reduces response variance in target languages via simple prompt engineering, yielding consistent 20–25% accuracy improvements across multiple LLMs and effectively mitigating the cross-lingual gap.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Language and Vision

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Any piece of knowledge is usually expressed in one or a handful of natural languages on the web or in any large corpus. Large Language Models (LLMs) act as a bridge by acquiring knowledge from a source language and making it accessible when queried from target languages. Prior research has pointed to a cross-lingual gap, viz., a drop in accuracy when the knowledge is queried in a target language compared to when the query is in the source language. Existing research has rationalized divergence in latent representations in source and target languages as the source of cross-lingual gap. In this work, we take an alternative view and hypothesize that the variance of responses in the target language is the main cause of this gap. For the first time, we formalize the cross-lingual gap in terms of bias-variance decomposition. We present extensive experimental evidence which support proposed formulation and hypothesis. We then reinforce our hypothesis through multiple inference-time interventions that control the variance and reduce the cross-lingual gap. We demonstrate a simple prompt instruction to reduce the response variance, which improved target accuracy by 20-25% across different models.

Problem

• Investigating cross-lingual performance gaps in Large Language Models
• Proposing variance in target language responses causes accuracy drop
• Developing interventions to reduce variance and improve target accuracy

Innovation

• Formalized cross-lingual gap via bias-variance decomposition
• Identified response variance as primary cause of accuracy drop
• Used prompt instructions to reduce variance and improve accuracy