Echoes of Power: Investigating Geopolitical Bias in US and China Large Language Models

2025-03-20

AI Summary

This study investigates ideological and cultural biases embedded in large language models (LLMs) regarding geopolitics, specifically comparing ChatGPT (U.S.-developed) and DeepSeek (China-developed). Method: Leveraging a curated geopolitical question set, we conduct a multimodal evaluation—integrating qualitative discourse analysis with quantitative metrics including stance polarity, factual density, and rhetorical bias—to assess cross-model consistency on sensitive political topics. Contribution/Results: Our analysis reveals significant yet non-binary divergence: the models exhibit high response agreement on 42% of questions, challenging the assumption of technological determinism in geopolitical alignment. Crucially, they demonstrate substantial convergence on foundational factual claims, indicating latent capacity for cross-ideological factual alignment. This work provides the first empirical, multidimensional assessment of mainstream U.S. and Chinese LLMs on geopolitical discourse, offering both methodological innovation—via integrated qualitative-quantitative evaluation—and empirical grounding for AI value embedding research and transnational LLM governance frameworks.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Philosophy and Ethics of AI: Bias, Fairness & Equity

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Large Language Models (LLMs) have emerged as powerful tools for generating human-like text, transforming human-machine interactions. However, their widespread adoption has raised concerns about their potential to influence public opinion and shape political narratives. In this work, we investigate the geopolitical biases in US and Chinese LLMs, focusing on how these models respond to questions related to geopolitics and international relations. We collected responses from ChatGPT and DeepSeek to a set of geopolitical questions and evaluated their outputs through both qualitative and quantitative analyses. Our findings show notable biases in both models, reflecting distinct ideological perspectives and cultural influences. However, despite these biases, for a set of questions, the models' responses are more aligned than expected, indicating that they can address sensitive topics without necessarily presenting directly opposing viewpoints. This study highlights the potential of LLMs to shape public discourse and underscores the importance of critically assessing AI-generated content, particularly in politically sensitive contexts.

Problem

• Investigating geopolitical biases in US and Chinese LLMs
• Comparing responses to geopolitical questions from ChatGPT and DeepSeek
• Assessing AI-generated content in politically sensitive contexts

Innovation

• Analyzing geopolitical biases in US and Chinese LLMs
• Qualitative and quantitative evaluation of model responses
• Identifying alignment in responses on sensitive topics