PR2: A Physics‐ and Photo‐Realistic Humanoid Testbed With Pilot Study in Competition

2024-09-03

Journal of Field Robotics

AI Summary

Embodied AI and robotics research lacks high-fidelity, scalable testbeds for evaluating humanoid robots. Method: This work introduces PR2—a physically accurate, photorealistic, full-scale humanoid robot simulation platform—integrating a high-precision rigid-body dynamics engine with a real-time rendering framework to support multi-sensor simulation, large language model (LLM) interfaces, closed-loop motion planning, and control. It proposes the first comprehensive benchmark suite covering gait generation, dexterous locomotion-manipulation (loco-manipulation), and language-guided navigation. Contribution/Results: The benchmark was deployed in a national undergraduate humanoid robotics competition, enabling 137 university teams to evaluate performance on walking, loco-manipulation, and language-driven search tasks. Empirical results validate the platform’s efficacy across diverse embodied intelligence capabilities. Following open-sourcing, PR2 has significantly accelerated pedagogical adoption and algorithmic development in embodied AI and robotics.

Technology Category

• Intelligent Robots: Embodied AI
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

This paper presents the development of a Physics‐realistic and Photo‐realistic humanoid robot testbed, PR2, to facilitate collaborative research between Embodied Artificial Intelligence (Embodied AI) and robotics. PR2 offers high‐quality scene rendering and robot dynamic simulation, enabling (i) the creation of diverse scenes using various digital assets, (ii) the integration of advanced perception or foundation models, and (iii) the implementation of planning and control algorithms for dynamic humanoid robot behaviors based on environmental feedback. The beta version of PR2 has been deployed for the simulation track of a nationwide full‐size humanoid robot competition for college students, attracting 137 teams and over 400 participants within 4 months. This competition covered traditional tasks in bipedal walking, as well as novel challenges in loco‐manipulation and language‐instruction‐based object search, marking a first for public college robotics competitions. A retrospective analysis of the competition suggests that future events should emphasize the integration of locomotion with manipulation and perception. By making the PR2 testbed publicly available at https://github.com/pr2-humanoid/PR2-Platform, we aim to further advance education and training in humanoid robotics. Video demonstration: https://pr2-humanoid.github.io/.

Problem

• Develops a physics- and photo-realistic humanoid robot testbed for research
• Enables dynamic humanoid behaviors via perception, planning, and control integration
• Supports education and competition in humanoid robotics with public platform

Innovation

• High-quality scene rendering and dynamic simulation
• Integration of perception and foundation models
• Planning and control for dynamic humanoid behaviors