To Lie or Not to Lie? Investigating The Biased Spread of Global Lies by LLMs

2026-04-07

AI Summary

This study addresses significant geographic and linguistic biases in large language models (LLMs) when generating disinformation, particularly their inadequate safeguards for countries with low Human Development Index (HDI) scores and low-resource languages. The authors introduce GlobalLies, the first multilingual parallel dataset for global disinformation research, spanning eight languages and 195 countries. By integrating human annotations, LLM-as-a-judge evaluations, input safety classifiers, and retrieval-augmented fact-checking, the work systematically uncovers LLMs’ varying propensities to generate false information across linguistic and national contexts. For the first time, this research quantifies and publicly discloses such systemic biases and releases accompanying multilingual prompt templates and an entity-level dataset, establishing a benchmark and toolkit for cross-lingual disinformation mitigation.

Abstract

Misinformation is on the rise, and the strong writing capabilities of LLMs lower the barrier for malicious actors to produce and disseminate false information. We study how LLMs behave when prompted to spread misinformation across languages and target countries, and introduce GlobalLies, a multilingual parallel dataset of 440 misinformation generation prompt templates and 6,867 entities, spanning 8 languages and 195 countries. Using both human annotations and large-scale LLM-as-a-judge evaluations across hundreds of thousands of generations from state-of-the-art models, we show that misinformation generation varies systematically based on the country being discussed. Propagation of lies by LLMs is substantially higher in many lower-resource languages and for countries with a lower Human Development Index (HDI). We find that existing mitigation strategies provide uneven protection: input safety classifiers exhibit cross-lingual gaps, and retrieval-augmented fact-checking remains inconsistent across regions due to unequal information availability. We release GlobalLies for research purposes, aiming to support the development of mitigation strategies to reduce the spread of global misinformation: https://github.com/zohaib-khan5040/globallies

Problem

• misinformation
• large language models
• cross-lingual bias
• global lies
• Human Development Index

Innovation

• GlobalLies
• multilingual misinformation
• LLM bias
• cross-lingual safety
• retrieval-augmented fact-checking