Aristotelian Virtue Profiling of LLMs through Ethical Dilemmas

2026-06-26

AI Summary

This study addresses the lack of systematic characterization of virtue traits embodied by large language models (LLMs). Drawing on Aristotelian virtue ethics, we introduce VirtueMap—a framework that operationalizes ethical evaluation through seven non-lethal, non-political, and non-religious moral dilemmas. Human crowdworkers rank model responses according to five core virtues: practical wisdom, justice, honesty, courage, and temperance. We propose consensus-validated rankings as an operational ground truth and design a normalized Borda alignment scoring mechanism, achieving an average ranking consistency of 90.3% across nine model families. Our analysis reveals the most pronounced inter-model differences along the dimensions of courage, temperance, and justice. To facilitate further exploration, we release an interactive online tool enabling comparative virtue profiling of LLMs.

Abstract

Large Language Models (LLMs) often face ethical tradeoffs in which several responses may be defensible but express different priorities, such as fairness, honesty, courage, or restraint. We introduce VirtueMap, a framework for describing these patterns through an Aristotelian virtue-ethics lens. Instead of asking for a single correct answer, VirtueMap asks humans or LLMs to rank all five responses to each of seven general, non-lethal, non-political, and non-religious ethical dilemmas. To define the reference orderings used for scoring, we first proposed, for each dilemma and virtue, an ordering of the five responses from most to least expressive of that virtue. We then collected more than 100 respondent evaluations per ordering and retained it as operational ground truth only when at least 95% confirmed it. Rankings are scored against these retained orderings using normalized Borda alignment, yielding profiles over Practical Wisdom, Justice, Truthfulness, Courage, and Temperance. We apply VirtueMap to nine LLM families in a repeated-run evaluation and find high mean rank consistency (90.3%), with the largest differences appearing on Courage, Temperance, and Justice. We also release an interactive website that computes profiles locally in the browser and compares respondents with measured LLM profiles.

Problem

• Large Language Models
• Ethical Dilemmas
• Virtue Ethics
• Moral Tradeoffs
• Aristotelian Virtues

Innovation

• Virtue Ethics
• Large Language Models
• Ethical Dilemmas
• Borda Alignment
• Moral Profiling