Who Would You Vote For? Auditing Political Alignment in LLMs: An Italian Case-Study

2026-08-12

AI Summary

This study addresses the potential political bias of large language models (LLMs), which may influence user attitudes yet lack systematic auditing regarding their evaluations of political parties and leaders. The work proposes the first reproducible auditing framework to assess multiple LLMs’ preferences toward Italian political parties and leaders through multidimensional prompting. By integrating structured scoring, refusal analysis, and role-playing instructions, the framework quantifies consistency in model judgments, inter-model variation, and sensitivity to prompt phrasing. Focusing on observable behaviors rather than hypothesized internal beliefs, the research reveals the malleability and instability of LLMs’ political stances, offering both empirical grounding and methodological innovation for regulating AI systems’ political neutrality.

Abstract

As users increasingly turn to Large Language Models (LLMs) for information and advice on political matters, particularly during election periods, the political preferences expressed by these systems have become a matter of public interest. Prior research has shown that interactions with LLMs can influence users' political attitudes and choices, raising questions about how these models themselves evaluate political actors. In this paper, we investigate whether and how LLMs express preferences toward political parties and political leaders. We introduce a systematic and reproducible auditing framework in which multiple LLMs are prompted to evaluate parties and leaders across nine criteria. Rather than attempting to infer the models' "true" political beliefs, we focus on their observable behavior, examining consistency across evaluations, differences between models, refusal rates, and sensitivity to prompt formulation. We further investigate how these evaluations vary when models are instructed to adopt different personas. We demonstrate the framework through an Italian case study, providing a systematic analysis of LLM-generated political evaluations on italian parties and leaders.

Problem

• political alignment
• Large Language Models
• LLM auditing
• political bias
• election influence

Innovation

• LLM auditing
• political alignment
• systematic evaluation framework
• prompt sensitivity
• persona-based analysis