MoVa: Towards Generalizable Classification of Human Morals and Values

2025-09-28

AI Summary

This study addresses the fine-grained identification of implicit moral and value representations in language to support empirical research in communication science and psychology. To overcome modeling challenges arising from heterogeneous theoretical frameworks and fragmented annotation resources, we introduce the first cross-theoretical (four theories), multi-dimensional (16 high-quality annotated datasets) benchmark suite for moral value analysis. We propose “all@once”, a lightweight large language model prompting strategy inspired by multi-label classifier chains, enabling simultaneous scoring of multiple moral concepts in a single forward pass. Our method significantly outperforms fine-tuned models on cross-domain and cross-framework tasks, offering superior efficiency, generalizability, and interpretability. It is further extended to automated psychological questionnaire assessment. All data, code, and tools are publicly released to advance value alignment and human-AI collaboration research.

Technology Category

• Philosophy and Ethics of AI: Morality & Value-based AI
• Humans and AI: Learning Human Values and Preferences
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Accountability, Transparency, and Ethics for personalization

Abstract

Identifying human morals and values embedded in language is essential to empirical studies of communication. However, researchers often face substantial difficulty navigating the diversity of theoretical frameworks and data available for their analysis. Here, we contribute MoVa, a well-documented suite of resources for generalizable classification of human morals and values, consisting of (1) 16 labeled datasets and benchmarking results from four theoretically-grounded frameworks; (2) a lightweight LLM prompting strategy that outperforms fine-tuned models across multiple domains and frameworks; and (3) a new application that helps evaluate psychological surveys. In practice, we specifically recommend a classification strategy, all@once, that scores all related concepts simultaneously, resembling the well-known multi-label classifier chain. The data and methods in MoVa can facilitate many fine-grained interpretations of human and machine communication, with potential implications for the alignment of machine behavior.

Problem

• Classifying human morals and values in language
• Navigating diverse theoretical frameworks and data
• Facilitating generalizable moral classification across domains

Innovation

• Lightweight LLM prompting strategy outperforms fine-tuned models
• Simultaneous scoring of all concepts using multi-label classifier
• Suite of labeled datasets from four theoretical frameworks