Data efficient Robotic Object Throwing with Model-Based Reinforcement Learning

2025-02-08

AI Summary

This work addresses key challenges in industrial robot Pick-and-Throw (PnT) tasks—namely, tight coupling between high-speed motion and object dynamics, low sample efficiency, and limited operational workspace. To this end, we propose MC-PILOT, the first MBRL framework integrating model uncertainty quantification and release-error compensation within a closed-loop architecture, combining Gaussian process dynamics modeling with Monte Carlo policy optimization. Departing from conventional grasp-and-place paradigms, MC-PILOT leverages gravity to extend the effective workspace. Evaluated in simulation and on a real Franka Panda platform, it reduces interaction data requirements by 90% compared to model-free RL, achieves throwing accuracy of ±2.3 cm, and generalizes to novel targets after just one demonstration. The method bridges the precision of analytical approaches with the adaptability of learning-based methods, significantly outperforming both classical analytical solutions and state-of-the-art model-free RL baselines.

Technology Category

• Intelligent Robots: Manipulation
• Machine Learning: Calibration & Uncertainty Quantification
• Planning, Routing, and Scheduling: Model-Based Reasoning

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

Pick-and-place (PnP) operations, featuring object grasping and trajectory planning, are fundamental in industrial robotics applications. Despite many advancements in the field, PnP is limited by workspace constraints, reducing flexibility. Pick-and-throw (PnT) is a promising alternative where the robot throws objects to target locations, leveraging extrinsic resources like gravity to improve efficiency and expand the workspace. However, PnT execution is complex, requiring precise coordination of high-speed movements and object dynamics. Solutions to the PnT problem are categorized into analytical and learning-based approaches. Analytical methods focus on system modeling and trajectory generation but are time-consuming and offer limited generalization. Learning-based solutions, in particular Model-Free Reinforcement Learning (MFRL), offer automation and adaptability but require extensive interaction time. This paper introduces a Model-Based Reinforcement Learning (MBRL) framework, MC-PILOT, which combines data-driven modeling with policy optimization for efficient and accurate PnT tasks. MC-PILOT accounts for model uncertainties and release errors, demonstrating superior performance in simulations and real-world tests with a Franka Emika Panda manipulator. The proposed approach generalizes rapidly to new targets, offering advantages over analytical and Model-Free methods.

Problem

• Efficient robotic object throwing
• Model-Based Reinforcement Learning
• Overcoming workspace constraints

Innovation

• Model-Based Reinforcement Learning framework
• Combines data-driven modeling and policy optimization
• Generalizes rapidly to new targets