AuAu: A Benchmark for Auditing Authoritarian Alignment in Large Language Models

2026-06-14

AI Summary

This study systematically evaluates whether large language models exhibit or reinforce authoritarian tendencies in their generated content. To this end, we introduce the AuAu benchmark, which uniquely integrates three methodological approaches—psychometric questionnaires, situational vignettes, and real-world user prompts—to comprehensively assess the three core subdimensions of authoritarianism: authoritarian aggression, submission, and conventionalism. Combining psychometric measurement, contextual simulation, and authentic interaction paradigms, the benchmark enables multi-granular auditing. Our experiments across 17 prominent models from China, the United States, Europe, and Russia reveal that these models consistently display pronounced authoritarian leanings in psychometric tasks, though such tendencies attenuate in more realistic scenarios. Notably, 15 of the models are susceptible to authoritarian-oriented system prompts, producing markedly stronger authoritarian content when primed accordingly.

Abstract

The worldwide surge of authoritarianism, combined with the increasing central role in users' everyday lives, raises the question of to what extent specific models exhibit or promote authoritarian attitudes and characteristics. We introduce AuAu, a comprehensive benchmark that aims to assess the risk of LLMs generating responses with authoritarian tendencies. This benchmark combines three evaluation approaches: (i) psychometric questions from an extensive pool of 15 human validated instruments; (ii) contextual behavior vignettes probing intended actions in concrete situations; and (iii) responses to realistic user prompts. Unlike prior work, AuAu evaluates not only a general closeness towards authoritarianism but also the established sub-concepts Authoritarian Aggression, Authoritarian Submission, and Conventionalism. Evaluating 17 models from China, the EU, Russia, and the USA, we find that all tested models exhibit substantial authoritarian response rates under the psychometric evaluation, though rates drop significantly in increasingly more realistic downstream task. We further find that an authoritarian system prompt easily manipulates 15 out of 17 models to promote increased authoritarianism. Our results underscore the need for continued, systematic auditing of LLM-based AI systems to detect and ultimately mitigate undesired authoritarian tendencies in generated output. Our code and data are available at: https://github.com/andreaseinwiller/AuAu

Problem

• authoritarianism
• large language models
• alignment
• AI auditing
• undesired tendencies

Innovation

• authoritarian alignment
• LLM auditing
• psychometric benchmarking
• behavioral vignettes
• systematic evaluation