PARC: Physics-based Augmentation with Reinforcement Learning for Character Controllers

2025-05-06

AI Summary

Motion capture data for agile locomotion over complex terrains is scarce, hindering robust controller training. Method: This paper proposes a physics-guided, closed-loop data augmentation framework: a motion generator is initialized with minimal expert demonstrations and iteratively refined via online physics simulation and reinforcement learning to synthesize and correct locomotion trajectories on challenging terrains. Contact-aware trajectory optimization and a physics-based tracking controller eliminate motion artifacts; corrected trajectories are fed back into the dataset for iterative distillation. Contribution/Results: Our approach enables the first joint co-evolution of motion generator and tracking controller without requiring real-world motion capture. Experiments demonstrate substantial improvements in success rate and motion naturalness on high-dynamic tasks—including wall climbing and gap crossing—while achieving generalization performance comparable to models trained on large-scale mocap datasets, despite limited initial data.

Technology Category

• Computer Vision: Motion & Tracking
• Intelligent Robots: Motion and Path Planning
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Humans excel in navigating diverse, complex environments with agile motor skills, exemplified by parkour practitioners performing dynamic maneuvers, such as climbing up walls and jumping across gaps. Reproducing these agile movements with simulated characters remains challenging, in part due to the scarcity of motion capture data for agile terrain traversal behaviors and the high cost of acquiring such data. In this work, we introduce PARC (Physics-based Augmentation with Reinforcement Learning for Character Controllers), a framework that leverages machine learning and physics-based simulation to iteratively augment motion datasets and expand the capabilities of terrain traversal controllers. PARC begins by training a motion generator on a small dataset consisting of core terrain traversal skills. The motion generator is then used to produce synthetic data for traversing new terrains. However, these generated motions often exhibit artifacts, such as incorrect contacts or discontinuities. To correct these artifacts, we train a physics-based tracking controller to imitate the motions in simulation. The corrected motions are then added to the dataset, which is used to continue training the motion generator in the next iteration. PARC's iterative process jointly expands the capabilities of the motion generator and tracker, creating agile and versatile models for interacting with complex environments. PARC provides an effective approach to develop controllers for agile terrain traversal, which bridges the gap between the scarcity of motion data and the need for versatile character controllers.

Problem

• Overcoming scarcity of motion data for agile terrain traversal
• Correcting artifacts in synthetic motion generation
• Developing versatile controllers for complex environments

Innovation

• Combines reinforcement learning with physics-based simulation
• Iteratively augments motion datasets for terrain traversal
• Corrects motion artifacts using physics-based tracking controller