Can LLMs Imagine Moral Alternatives Beyond Binary Dilemmas?

2026-06-30

AI Summary

This study addresses the limitation of current large language models (LLMs) in moral reasoning, which often restrict responses to binary choices and lack the capacity to generate alternative resolutions. The authors introduce MoralAltDataset, comprising 307 moral dilemmas, and propose a novel evaluation framework for non-binary moral alternatives—such as compromises and reframings—that integrates LLM-generated solutions, human preference experiments, expert ethical assessments, and structured judgment analysis. Findings reveal that such alternatives significantly shift moral decision tendencies; moreover, LLM-generated alternatives frequently surpass human-written ones in ethical soundness and structural quality, though they entail trade-offs in practical feasibility. This work thus illuminates both the promise and constraints of LLMs in navigating complex moral reasoning beyond dichotomous choices.

Abstract

As large language models (LLMs) are increasingly deployed as moral advisors and agents, they need to address dilemmas between two competing values. However, existing research on LLMs with moral dilemmas overlooks a central aspect of human moral cognition: the ability to imagine alternatives that move beyond the given options. We introduce MoralAltDataset, a dataset of 307 moral dilemmas spanning narrative Advisor dilemmas and AI-facing Agent dilemmas, each augmented with compromise and reframed alternatives. We first examine whether humans and LLMs shift their judgments when such alternatives are introduced. Across 15 LLMs, we find that compromise alternatives are often preferred over either original option, substantially reshaping moral choice. We then evaluate the quality of LLM-generated alternatives against human-authored ones using pairwise preference and expert-based criteria. Results show that LLM-generated alternatives are often preferred and better satisfy fine-grained structural and ethical criteria, while revealing trade-offs between structural quality and practical feasibility.

Problem

• moral dilemmas
• large language models
• moral alternatives
• compromise
• reframing

Innovation

• moral alternatives
• large language models
• moral dilemmas
• compromise generation
• ethical reasoning