Directing the Robot: Scaffolding Creative Human-AI-Robot Interaction

2026-03-08

AI Summary

Current human–AI–robot interaction paradigms often prioritize efficiency, relegating humans to a supervisory role and struggling to support open-ended, improvisational collaboration. This work proposes a “scaffolding” interaction framework that positions AI as a mediating layer between human intent and robotic execution, enabling users to act as “executive directors” who continuously guide and dynamically shape robot behavior. Grounded in user-centered design and informed by contexts such as creative co-creation and pedagogy, the approach facilitates flexible expression of intent and real-time adaptation. Empirical validation demonstrates its feasibility in fostering creativity, autonomy, and flow experiences in open-ended collaborative scenarios. The study further introduces novel evaluation dimensions tailored to collaborative creation and highlights key challenges for future work, including social intelligence and system scalability.

Technology Category

• Intelligent Robots: Human-Robot Interaction
• Humans and AI: Interaction Techniques and Devices
• Cognitive Modeling & Cognitive Systems: Computational Creativity

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Research challenges in human and human-AI computation

Abstract

Robots are moving beyond industrial settings into creative, educational, and public environments where interaction is open-ended and improvisational. Yet much of human-AI-robot interaction remains framed around performance and efficiency, positioning humans as supervisors rather than collaborators. We propose a re-framing of AI interaction with robots as scaffolding: infrastructure that enables humans to shape robotic behaviour over time while remaining meaningfully in control. Through scenarios from creative practice, learning-by-teaching, and embodied interaction, we illustrate how humans can act as executive directors, defining intent and steering revisions, while AI mediates between human expression and robotic execution. We outline design and evaluation implications that foreground creativity, agency, and flow. Finally, we discuss open challenges in social, scalable, and mission-critical contexts. We invite the community to rethink interacting with Robots and AI not as autonomy, but as sustained support for human creativity.

Problem

• human-AI-robot interaction
• creativity
• agency
• scaffolding
• collaboration

Innovation

• scaffolding
• human-AI-robot interaction
• creative collaboration
• agency
• embodied interaction