Language Bias under Conflicting Information in Multilingual LLMs

2026-04-08

AI Summary

This study investigates whether multilingual large language models exhibit linguistic preference biases when confronted with conflicting information presented in different languages. By extending the “needle-in-a-haystack” paradigm to multilingual settings—integrating real-world news data from five languages, a long-context evaluation framework, and a cross-lingual conflict injection methodology—the work systematically reveals, for the first time, the behavioral patterns of mainstream multilingual models (including GPT-5.2) in handling inter-lingual contradictions. The findings demonstrate that these models generally ignore conflicts and output a single response, consistently displaying a negative bias toward Russian and a positive bias toward Chinese in long-context scenarios—a phenomenon significantly present across both Chinese- and non-Chinese-trained models.

Abstract

Large Language Models (LLMs) have been shown to contain biases in the process of integrating conflicting information when answering questions. Here we ask whether such biases also exist with respect to which language is used for each conflicting piece of information. To answer this question, we extend the conflicting needles in a haystack paradigm to a multilingual setting and perform a comprehensive set of evaluations with naturalistic news domain data in five different languages, for a range of multilingual LLMs of different sizes. We find that all LLMs tested, including GPT-5.2, ignore the conflict and confidently assert only one of the possible answers in the large majority of cases. Furthermore, there is a consistent bias across models in which languages are preferred, with a general bias against Russian and, for the longest context lengths, in favor of Chinese. Both of these patterns are consistent between models trained inside and outside of mainland China, though somewhat stronger in the former category.

Problem

• language bias
• conflicting information
• multilingual LLMs
• needle-in-a-haystack
• cross-lingual preference

Innovation

• multilingual LLMs
• language bias
• conflicting information
• needle-in-a-haystack
• cross-lingual evaluation