Prompt2Auto: From Motion Prompt to Automated Control via Geometry-Invariant One-Shot Gaussian Process Learning

2025-09-17

AI Summary

Conventional demonstration-based robotic learning relies heavily on large-scale labeled datasets and suffers from poor generalization under coordinate transformations—such as translation, rotation, and scaling—failing to achieve robust multi-step autonomous control from a single human demonstration. Method: We propose GeoGP, a geometrically invariant one-shot Gaussian process learning framework. It introduces a coordinate-transformation-aware data construction strategy, integrates geometrically invariant kernel functions with Gaussian process regression, and incorporates a graphical user interface for both simulation and real-robot validation. Contribution/Results: Evaluated in simulation and on two physical robotic arms, GeoGP achieves cross-coordinate-system, multi-skill, and multi-step motion prediction from only one demonstration. It significantly reduces annotation effort, exhibits strong cross-task generalization, and maintains robustness against user input perturbations—establishing a novel paradigm for low-sample, high-generalization robotic skill acquisition.

Technology Category

• Intelligent Robots: Learning & Optimization for ROB
• Machine Learning: Learning with Manifolds
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Learning from demonstration allows robots to acquire complex skills from human demonstrations, but conventional approaches often require large datasets and fail to generalize across coordinate transformations. In this paper, we propose Prompt2Auto, a geometry-invariant one-shot Gaussian process (GeoGP) learning framework that enables robots to perform human-guided automated control from a single motion prompt. A dataset-construction strategy based on coordinate transformations is introduced that enforces invariance to translation, rotation, and scaling, while supporting multi-step predictions. Moreover, GeoGP is robust to variations in the user's motion prompt and supports multi-skill autonomy. We validate the proposed approach through numerical simulations with the designed user graphical interface and two real-world robotic experiments, which demonstrate that the proposed method is effective, generalizes across tasks, and significantly reduces the demonstration burden. Project page is available at: https://prompt2auto.github.io

Problem

• Enables robot control from single motion prompt
• Achieves invariance to translation, rotation, and scaling
• Reduces demonstration burden while supporting multi-skill autonomy

Innovation

• Geometry-invariant Gaussian process learning
• One-shot automated control from motion prompt
• Coordinate transformation-based dataset construction