Preserving Sense of Agency: User Preferences for Robot Autonomy and User Control across Household Tasks

2025-06-23

AI Summary

This study investigates how the autonomy level of domestic assistant robots affects users’ sense of agency (i.e., perceived environmental control) and their preferences for control allocation across varying risk contexts. Employing a four-level autonomy experimental design, we conducted empirical evaluations across diverse household tasks, collecting user ratings and task-specific control-preference rankings. Results indicate that users strongly prefer direct control in high-risk scenarios; third-party intervention significantly diminishes perceived agency, whereas end-user programmable robots sustain higher agency. Crucially, users’ trust in third-party operators moderates autonomy preferences. The findings reveal a dynamic equilibrium between robotic autonomy and human control, underscoring user programmability as a core design principle. This work advances theoretical understanding of human-robot co-control and provides empirically grounded guidelines for designing domestic robots that simultaneously ensure safety and preserve user agency.

Technology Category

• Intelligent Robots: Human-Robot Interaction
• Humans and AI: Learning Human Values and Preferences
• Knowledge Representation and Reasoning: Preferences

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

Roboticists often design with the assumption that assistive robots should be fully autonomous. However, it remains unclear whether users prefer highly autonomous robots, as prior work in assistive robotics suggests otherwise. High robot autonomy can reduce the user's sense of agency, which represents feeling in control of one's environment. How much control do users, in fact, want over the actions of robots used for in-home assistance? We investigate how robot autonomy levels affect users' sense of agency and the autonomy level they prefer in contexts with varying risks. Our study asked participants to rate their sense of agency as robot users across four distinct autonomy levels and ranked their robot preferences with respect to various household tasks. Our findings revealed that participants' sense of agency was primarily influenced by two factors: (1) whether the robot acts autonomously, and (2) whether a third party is involved in the robot's programming or operation. Notably, an end-user programmed robot highly preserved users' sense of agency, even though it acts autonomously. However, in high-risk settings, e.g., preparing a snack for a child with allergies, they preferred robots that prioritized their control significantly more. Additional contextual factors, such as trust in a third party operator, also shaped their preferences.

Problem

• Investigates user preferences for robot autonomy versus control in household tasks
• Examines how autonomy levels impact users' sense of agency
• Explores autonomy preferences in high-risk versus low-risk scenarios

Innovation

• Adjustable robot autonomy based on user preference
• End-user programmed robots enhance sense of agency
• Context-aware autonomy prioritizes control in high-risk tasks