Uncertain Machine Ethics Planning

2025-05-07

AI Summary

This work addresses the challenge of modeling uncertainty and reconciling conflicts among heterogeneous moral theories—utilitarianism, deontology, and virtue ethics—in autonomous ethical decision-making. We propose the Multi-Moral Markov Decision Process (MM-MDP) and its instantiation as a Multi-Moral Stochastic Shortest Path problem. Our approach innovatively integrates Hansson’s hypothetical backward reasoning mechanism to enable interpretable, uncertainty-aware ethical planning; combines multi-objective AO* search with cross-theoretical utility normalization, moral representation mapping, and conflict-resolution strategies. Evaluated on the canonical “insulin theft” case, our method generates robust, long-horizon policies satisfying constraints from all three moral frameworks, outperforming baselines in moral consistency, traceable reasoning, and computational feasibility. To our knowledge, this is the first framework enabling unified, formally verifiable sequential ethical decision-making across heterogeneous moral theories.

Technology Category

• Reasoning under Uncertainty: Sequential Decision Making
• Philosophy and Ethics of AI: Morality & Value-based AI
• Multiagent Systems: Multiagent Planning

Application Category

• User Modeling, Personalization and Recommendation: Accountability, Transparency, and Ethics for personalization
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Fairness and ethical considerations in crowd work and in human-in-the-loop AI systems

Abstract

Machine Ethics decisions should consider the implications of uncertainty over decisions. Decisions should be made over sequences of actions to reach preferable outcomes long term. The evaluation of outcomes, however, may invoke one or more moral theories, which might have conflicting judgements. Each theory will require differing representations of the ethical situation. For example, Utilitarianism measures numerical values, Deontology analyses duties, and Virtue Ethics emphasises moral character. While balancing potentially conflicting moral considerations, decisions may need to be made, for example, to achieve morally neutral goals with minimal costs. In this paper, we formalise the problem as a Multi-Moral Markov Decision Process and a Multi-Moral Stochastic Shortest Path Problem. We develop a heuristic algorithm based on Multi-Objective AO*, utilising Sven-Ove Hansson's Hypothetical Retrospection procedure for ethical reasoning under uncertainty. Our approach is validated by a case study from Machine Ethics literature: the problem of whether to steal insulin for someone who needs it.

Problem

• Addressing uncertainty in Machine Ethics decision-making processes
• Balancing conflicting moral theories in sequential action planning
• Formalizing ethical dilemmas as multi-moral stochastic path problems

Innovation

• Multi-Moral Markov Decision Process formalization
• Heuristic algorithm based on Multi-Objective AO*
• Utilizes Hypothetical Retrospection for ethical reasoning