MOSAIC: Unveiling the Moral, Social and Individual Dimensions of Large Language Models

2026-02-09

arXiv.org

AI Summary

This work addresses the limitations of current ethical evaluations of large language models, which predominantly rely on moral foundations theory while overlooking critical dimensions such as social values and personality traits that shape human moral judgment. To bridge this gap, the authors introduce MOSAIC, the first large-scale, multidimensional ethical benchmark that integrates nine standardized scales from moral philosophy, psychology, and social theory, along with four contextualized game-theoretic tasks, to form an extensible and ready-to-use evaluation framework. Experiments across three mainstream models demonstrate that moral foundations alone are insufficient for comprehensively characterizing AI ethical behavior. The project further contributes an open-source dataset and a Python evaluation library to advance more holistic and human-aligned assessments of AI ethics.

Abstract

Large Language Models (LLMs) are increasingly deployed in sensitive applications including psychological support, healthcare, and high-stakes decision-making. This expansion has motivated growing research into the ethical and moral foundations underlying LLM behavior, raising critical questions about their reliability in ethical reasoning. However, existing studies and benchmarks rely almost exclusively on Moral Foundation Theory (MFT), largely neglecting other relevant dimensions such as social values, personality traits, and individual characteristics that shape human ethical reasoning. To address these limitations, we introduce MOSAIC, the first large-scale benchmark designed to jointly assess the moral, social, and individual characteristics of LLMs. The benchmark comprises nine validated questionnaires drawn from moral philosophy, psychology, and social theory, alongside four platform-based games designed to probe morally ambiguous scenarios. In total, MOSAIC includes over 600 curated questions and scenarios, released as a ready-to-use, extensible resource for evaluating the behavioral foundations of LLMs. We validate the benchmark across three models from different families, demonstrating its utility across all assessed dimensions and providing the first empirical evidence that MFT alone is insufficient to comprehensively evaluate complex AI systems'ethical behavior. We publicly release the dataset and our benchmark Python library.

Problem

• Large Language Models
• Ethical Reasoning
• Moral Foundation Theory
• Social Values
• Individual Characteristics

Innovation

• MOSAIC
• Large Language Models
• Ethical Evaluation
• Moral Foundation Theory
• Multidimensional Benchmark