Active Robot Curriculum Learning from Online Human Demonstrations

2025-03-04

AI Summary

This work addresses the high cognitive load and low demonstration quality incurred by non-expert human demonstrators in active Learning from Demonstration (LfD), caused by frequent context switching during teaching. We propose the first curriculum-based active demonstration querying framework, which guides humans to provide demonstrations online in increasing order of task difficulty, jointly optimizing robot sample efficiency and human teaching performance. Evaluated on four sparse-reward simulated tasks, our method significantly improves policy convergence success rates and sample efficiency. A user study (N=26) shows a 37% reduction in demonstration time, a 52% decrease in failure rate, and markedly enhanced teaching transferability and post-instruction adaptability. Our core contribution is the first human-centered curriculum design for active query selection—systematically improving teaching adaptability, transferability, and task success rate in concert.

Technology Category

• Machine Learning: Active Learning
• Intelligent Robots: Learning & Optimization for ROB
• Search and Optimization: Learning to Search

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Learning from Demonstrations (LfD) allows robots to learn skills from human users, but its effectiveness can suffer due to sub-optimal teaching, especially from untrained demonstrators. Active LfD aims to improve this by letting robots actively request demonstrations to enhance learning. However, this may lead to frequent context switches between various task situations, increasing the human cognitive load and introducing errors to demonstrations. Moreover, few prior studies in active LfD have examined how these active query strategies may impact human teaching in aspects beyond user experience, which can be crucial for developing algorithms that benefit both robot learning and human teaching. To tackle these challenges, we propose an active LfD method that optimizes the query sequence of online human demonstrations via Curriculum Learning (CL), where demonstrators are guided to provide demonstrations in situations of gradually increasing difficulty. We evaluate our method across four simulated robotic tasks with sparse rewards and conduct a user study (N=26) to investigate the influence of active LfD methods on human teaching regarding teaching performance, post-guidance teaching adaptivity, and teaching transferability. Our results show that our method significantly improves learning performance compared to three other LfD baselines in terms of the final success rate of the converged policy and sample efficiency. Additionally, results from our user study indicate that our method significantly reduces the time required from human demonstrators and decreases failed demonstration attempts. It also enhances post-guidance human teaching in both seen and unseen scenarios compared to another active LfD baseline, indicating enhanced teaching performance, greater post-guidance teaching adaptivity, and better teaching transferability achieved by our method.

Problem

• Optimizes query sequence for online human demonstrations using Curriculum Learning.
• Reduces human cognitive load and errors in active Learning from Demonstrations.
• Enhances robot learning and human teaching performance in sparse reward tasks.

Innovation

• Active LfD optimizes query sequence via Curriculum Learning
• Method reduces human time and failed demonstration attempts
• Enhances teaching adaptivity and transferability in unseen scenarios