🤖 AI Summary
This work addresses the limitations of conventional human feedback signals—namely, low discriminability, difficulty in quantification, and poor inter-user consistency—for teaching robots to non-expert users. We propose *progress* as a novel, task-agnostic feedback signal, formally defined and empirically validated as a normalized task completion percentage. This signal exhibits strong discriminability between success and failure, enables precise quantification of partial progress, identifies harmless but inefficient behaviors, achieves high cross-user consistency, and imposes zero additional annotation burden. We validate its efficacy and robustness through three empirical studies: an online crowdsourced experiment (N=76), an in-situ user study in public spaces (N=40), and a real-world ice cream topping task. Concurrently, we introduce and publicly release the first benchmark dataset of 40 novice demonstrations, annotated with exploratory actions and operational errors—designed explicitly to support progress-based learning.
📝 Abstract
Enhancing the expressiveness of human teaching is vital for both improving robots’ learning from humans and the human-teaching-robot experience. In this work, we characterize and test a little-used teaching signal: progress, designed to represent the completion percentage of a task. We conducted two online studies with 76 crowd-sourced participants and one public space study with 40 non-expert participants to validate the capability of this progress signal. We find that progress indicates whether the task is successfully performed, reflects the degree of task completion, identifies unproductive but harmless behaviors, and is likely to be more consistent across participants. Furthermore, our results show that giving progress does not require extra workload and time. An additional contribution of our work is a dataset of 40 non-expert demonstrations from the public space study through an ice cream topping-adding task, which we observe to be multi-policy and sub-optimal, with sub-optimality not only from teleoperation errors but also from exploratory actions and attempts. The dataset is available at https://github.com/TeachingwithProgress/Non-Expert_Demonstrations.