Gender and Political Bias in Large Language Models: A Demonstration Platform

2025-09-18

AI Summary

This study systematically evaluates gender and political group biases in large language models (LLMs) for predicting voting outcomes in the European Parliament. Method: We introduce ParlAI Vote, an interactive auditing platform that uniquely integrates parliamentary debate transcripts, roll-call vote records, and multidimensional demographic attributes of politicians—including gender, political group, and nationality—to jointly model vote prediction and gender classification, enable error decomposition, and support counterfactual analysis. Built upon the EuroParlVote benchmark, it combines LLM-based inference, structured data retrieval, and visualization techniques to ensure reproducible bias detection and attribution. Contribution/Results: Experiments reveal significant demographic disparities in state-of-the-art LLMs’ vote predictions. ParlAI Vote serves simultaneously as a model auditing tool, pedagogical demonstration framework, and public engagement platform—establishing a novel paradigm and open-source infrastructure for fairness assessment in political AI.

Technology Category

• Humans and AI: Voting
• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy
• Machine Learning: Ethics, Bias, and Fairness

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

We present ParlAI Vote, an interactive system for exploring European Parliament debates and votes, and for testing LLMs on vote prediction and bias analysis. This platform connects debate topics, speeches, and roll-call outcomes, and includes rich demographic data such as gender, age, country, and political group. Users can browse debates, inspect linked speeches, compare real voting outcomes with predictions from frontier LLMs, and view error breakdowns by demographic group. Visualizing the EuroParlVote benchmark and its core tasks of gender classification and vote prediction, ParlAI Vote highlights systematic performance bias in state-of-the-art LLMs. The system unifies data, models, and visual analytics in a single interface, lowering the barrier for reproducing findings, auditing behavior, and running counterfactual scenarios. It supports research, education, and public engagement with legislative decision-making, while making clear both the strengths and the limitations of current LLMs in political analysis.

Problem

• Analyzing gender and political bias in LLMs using European Parliament data
• Predicting voting outcomes and detecting systematic performance bias patterns
• Creating accessible platform for auditing LLM behavior in political analysis

Innovation

• Interactive platform for exploring parliamentary debates
• Connects debate data with demographic information and predictions
• Unifies data models and analytics for bias analysis