MoralBench: Moral Evaluation of LLMs

2024-06-06

arXiv.org

AI Summary

Existing benchmarks lack systematic evaluation of large language models’ (LLMs) moral reasoning capabilities, particularly across realistic, multidimensional ethical dilemmas. Method: This work introduces the first comprehensive benchmark for evaluating LLMs on real-world ethical reasoning, grounded in normative ethics theory. It proposes the “Moral Identity” assessment framework—integrating context sensitivity, value granularity, and cross-cultural ethical consensus—and constructs a high-quality, expert-annotated dataset. Evaluation employs multi-granularity scoring, adversarial case design, and collaborative validation by domain scholars. Contribution/Results: Empirical evaluation across mainstream LLMs reveals pervasive deficits: contextual insensitivity and value misalignment. The open-sourced dataset and codebase have been widely adopted in academia, establishing a reproducible, scalable paradigm for assessing LLM alignment with human moral standards.

Technology Category

• Philosophy and Ethics of AI: Morality & Value-based AI
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

In the rapidly evolving field of artificial intelligence, large language models (LLMs) have emerged as powerful tools for a myriad of applications, from natural language processing to decision-making support systems. However, as these models become increasingly integrated into societal frameworks, the imperative to ensure they operate within ethical and moral boundaries has never been more critical. This paper introduces a novel benchmark designed to measure and compare the moral reasoning capabilities of LLMs. We present the first comprehensive dataset specifically curated to probe the moral dimensions of LLM outputs, addressing a wide range of ethical dilemmas and scenarios reflective of real-world complexities. The main contribution of this work lies in the development of benchmark datasets and metrics for assessing the moral identity of LLMs, which accounts for nuance, contextual sensitivity, and alignment with human ethical standards. Our methodology involves a multi-faceted approach, combining quantitative analysis with qualitative insights from ethics scholars to ensure a thorough evaluation of model performance. By applying our benchmark across several leading LLMs, we uncover significant variations in moral reasoning capabilities of different models. These findings highlight the importance of considering moral reasoning in the development and evaluation of LLMs, as well as the need for ongoing research to address the biases and limitations uncovered in our study. We publicly release the benchmark at https://drive.google.com/drive/u/0/folders/1k93YZJserYc2CkqP8d4B3M3sgd3kA8W7 and also open-source the code of the project at https://github.com/agiresearch/MoralBench.

Problem

• Measure moral reasoning capabilities of LLMs
• Assess LLM alignment with human ethical standards
• Address biases in LLM moral evaluations

Innovation

• Novel benchmark for LLM moral evaluation
• Comprehensive dataset for ethical dilemmas
• Multi-faceted methodology combining quantitative and qualitative analysis