Autonomy by Design: Preserving Human Autonomy in AI Decision-Support

2025-06-30

AI Summary

This study examines the dual adverse effects of AI decision-support systems on human domain-specific autonomy—the capacity for self-directed action within professional or skill-based contexts—namely, erosion of skill competence and disruption of authentic value formation. Focusing on high-stakes domains including healthcare, finance, and education, we propose, for the first time, a socio-technical design framework centered on three pillars: (1) defect mechanism identification, (2) explicit role boundary delineation, and (3) support for reflective practice. Drawing on cross-domain empirical case studies, we integrate methods from human-computer interaction, AI architecture, and value-sensitive design to construct the first systematic normative framework for safeguarding domain-specific autonomy. The framework effectively mitigates AI-induced value incoherence and cognitive deskilling, thereby strengthening human agency and epistemic authority in high-level decision-making.

Technology Category

• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness
• Humans and AI: Learning Human Values and Preferences
• Application Domains: Humanities & Computational Social Science

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Fairness and ethical considerations in crowd work and in human-in-the-loop AI systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

AI systems increasingly support human decision-making across domains of professional, skill-based, and personal activity. While previous work has examined how AI might affect human autonomy globally, the effects of AI on domain-specific autonomy -- the capacity for self-governed action within defined realms of skill or expertise -- remain understudied. We analyze how AI decision-support systems affect two key components of domain-specific autonomy: skilled competence (the ability to make informed judgments within one's domain) and authentic value-formation (the capacity to form genuine domain-relevant values and preferences). By engaging with prior investigations and analyzing empirical cases across medical, financial, and educational domains, we demonstrate how the absence of reliable failure indicators and the potential for unconscious value shifts can erode domain-specific autonomy both immediately and over time. We then develop a constructive framework for autonomy-preserving AI support systems. We propose specific socio-technical design patterns -- including careful role specification, implementation of defeater mechanisms, and support for reflective practice -- that can help maintain domain-specific autonomy while leveraging AI capabilities. This framework provides concrete guidance for developing AI systems that enhance rather than diminish human agency within specialized domains of action.

Problem

• Examining AI's impact on domain-specific human autonomy
• Addressing erosion of skilled competence and authentic value-formation
• Developing autonomy-preserving AI design frameworks for specialized domains

Innovation

• Socio-technical design patterns for autonomy
• Defeater mechanisms to prevent autonomy erosion
• Reflective practice support in AI systems