Are We Aligned? A Preliminary Investigation of the Alignment of Responsible AI Values between LLMs and Human Judgment

2025-11-06

AI Summary

This study systematically evaluates the value alignment of 23 large language models (LLMs) with human judgments across four responsible AI tasks: value selection, importance scoring, trade-off decisions, and software requirement prioritization. It compares LLM outputs against two human reference groups—U.S. representative public samples and AI practitioners. Methodologically, it introduces the first quantitative, multi-dimensional evaluation framework for value alignment, incorporating cross-population benchmarking to identify task-specific biases. Results show that LLMs align more closely with AI practitioners’ values (e.g., fairness, privacy, transparency, safety, accountability) than with the general public’s, yet exhibit a significant behavioral fidelity gap between stated values and actual prioritization behavior. This inconsistency reveals a critical limitation in operationalizing ethical principles. The work provides empirical grounding and methodological guidance for fine-grained monitoring, benchmark development, and governance of LLM value alignment.

Technology Category

• Humans and AI: Learning Human Values and Preferences
• Philosophy and Ethics of AI: Morality & Value-based AI
• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Large Language Models (LLMs) are increasingly employed in software engineering tasks such as requirements elicitation, design, and evaluation, raising critical questions regarding their alignment with human judgments on responsible AI values. This study investigates how closely LLMs'value preferences align with those of two human groups: a US-representative sample and AI practitioners. We evaluate 23 LLMs across four tasks: (T1) selecting key responsible AI values, (T2) rating their importance in specific contexts, (T3) resolving trade-offs between competing values, and (T4) prioritizing software requirements that embody those values. The results show that LLMs generally align more closely with AI practitioners than with the US-representative sample, emphasizing fairness, privacy, transparency, safety, and accountability. However, inconsistencies appear between the values that LLMs claim to uphold (Tasks 1-3) and the way they prioritize requirements (Task 4), revealing gaps in faithfulness between stated and applied behavior. These findings highlight the practical risk of relying on LLMs in requirements engineering without human oversight and motivate the need for systematic approaches to benchmark, interpret, and monitor value alignment in AI-assisted software development.

Problem

• Investigating alignment of LLMs' responsible AI values with human judgments
• Evaluating LLMs' value preferences across selection and prioritization tasks
• Identifying gaps between stated values and applied behavior in requirements engineering

Innovation

• Evaluating LLM alignment with human responsible AI values
• Comparing value preferences across four contextual tasks
• Identifying gaps between stated values and applied behavior