Sample Efficient Robot Learning in Supervised Effect Prediction Tasks

2024-12-03

arXiv.org

AI Summary

Learning world models for robotics—particularly continuous, high-dimensional regression tasks such as action-effect prediction—suffers from low sample efficiency and prohibitively high real-world interaction costs. Method: This paper introduces MUSEL, the first systematic active learning framework tailored specifically for regression-based world model training. MUSEL decouples total uncertainty to precisely quantify model uncertainty and jointly optimizes sampling by integrating learning progress with input diversity. Unlike conventional classification-oriented active learning or intrinsic-motivation methods relying solely on learning progress, MUSEL rigorously adapts the active learning paradigm to regression settings. Results: In simulated desktop robotics experiments, MUSEL achieves significantly higher action-effect prediction accuracy with fewer real interactions, outperforming state-of-the-art baselines—including SVGP, IM, and LP—thereby reducing energy, time, and human labor costs.

Technology Category

• Machine Learning: Active Learning
• Reasoning under Uncertainty: Sequential Decision Making
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

In self-supervised robot learning, robots actively explore their environments and generate data by acting on entities in the environment. Therefore, an exploration policy is desired that ensures sample efficiency to minimize robot execution costs while still providing accurate learning. For this purpose, the robotic community has adopted Intrinsic Motivation (IM)-based approaches such as Learning Progress (LP). On the machine learning front, Active Learning (AL) has been used successfully, especially for classification tasks. In this work, we develop a novel AL framework geared towards robotics regression tasks, such as action-effect prediction and, more generally, for world model learning, which we call MUSEL - Model Uncertainty for Sample Efficient Learning. MUSEL aims to extract model uncertainty from the total uncertainty estimate given by a suitable learning engine by making use of earning progress and input diversity and use it to improve sample efficiency beyond the state-of-the-art action-effect prediction methods. We demonstrate the feasibility of our model by using a Stochastic Variational Gaussian Process (SVGP) as the learning engine and testing the system on a set of robotic experiments in simulation. The efficacy of MUSEL is demonstrated by comparing its performance to standard methods used in robot action-effect learning. In a robotic tabletop environment in which a robot manipulator is tasked with learning the effect of its actions, the experiments show that MUSEL facilitates higher accuracy in learning action effects while ensuring sample efficiency.

Problem

• Reduce costs in self-supervised robotic learning
• Improve sample efficiency in regression tasks
• Develop framework for action-effect prediction

Innovation

• Model Uncertainty for Sample-Efficient Learning (MUSEL)
• Combines predictive uncertainty, learning progress, diversity
• Uses SVDKL model for robotic tasks