Stability and Political Orientation of International LLMs: An Exploratory Multi-Run Study Conducted in French

2026-06-11

AI Summary

This study investigates the output stability and implicit ideological biases of international large language models (LLMs) in French-language political question answering. Drawing on the Political Compass questionnaire, the authors design a standardized experimental protocol in which 11 LLMs from diverse national and institutional backgrounds each generate 20 independent responses to 62 political statements. By integrating statistical consistency analysis with cross-model comparisons, this work introduces—for the first time in a French context—a multi-run evaluation framework to assess the stability of political stances and proposes a reproducible comparative assessment protocol. The results indicate that while models generally exhibit stable behavior, significant variations persist both across runs and between models, revealing the critical influence of model architecture, training data composition, and content moderation mechanisms on expressed political orientations.

Abstract

This study explores the stability and ideological orientation of political responses produced by various large language models (LLMs) in French. We designed a standardised experimental protocol based on a questionnaire inspired by the Political Compass, aimed at measuring the economic and socio-cultural positions of each model across 62 political statements. Eleven models from various organisations and countries were tested, each subjected to twenty independent runs to assess intra-model variability. The analysis focuses on response consistency, inter-model differences, and the presence of implicit political orientations. The results show that, despite a general degree of stability, significant variations appear from one run to the next and between models, reflecting the impact of architectures, training data, and moderation mechanisms. This study proposes a comparative evaluation protocol for LLMs in the context of political information and underscores the importance of accounting for implicit biases in the use of these systems.

Problem

• stability
• political orientation
• large language models
• bias
• multilingual LLMs

Innovation

• political bias
• LLM stability
• multi-run evaluation
• ideological orientation
• cross-model comparison