Development of a PPO-Reinforcement Learned Walking Tripedal Soft-Legged Robot using SOFA

2025-04-12

AI Summary

To address the challenge of autonomous navigation for tripedal soft robots on complex, rugged terrain, this paper proposes an end-to-end gait learning framework integrating the SOFA physics engine with Proximal Policy Optimization (PPO) reinforcement learning. We introduce the first deep customization of soft-body dynamics parameters within SOFA, tightly coupling them with PPO to enable high-fidelity real-time simulation and joint policy training. Our approach breaks from conventional single-target navigation paradigms by supporting multi-stage sequential goal tracking and millimeter-precision trajectory following—achieving a cumulative squared error of only 19 mm. The framework integrates real-time vision–physics co-simulation and a ROS-compatible control interface. Experiments demonstrate an 82% success rate in single-target reaching and maintain sequential navigation trajectory deviations consistently below 19 mm. All source code is publicly released.

Technology Category

• Intelligent Robots: Learning & Optimization for ROB
• Planning, Routing, and Scheduling: Replanning and Plan Repair
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Rigid robots were extensively researched, whereas soft robotics remains an underexplored field. Utilizing soft-legged robots in performing tasks as a replacement for human beings is an important stride to take, especially under harsh and hazardous conditions over rough terrain environments. For the demand to teach any robot how to behave in different scenarios, a real-time physical and visual simulation is essential. When it comes to soft robots specifically, a simulation framework is still an arduous problem that needs to be disclosed. Using the simulation open framework architecture (SOFA) is an advantageous step. However, neither SOFA's manual nor prior public SOFA projects show its maximum capabilities the users can reach. So, we resolved this by establishing customized settings and handling the framework components appropriately. Settling on perfect, fine-tuned SOFA parameters has stimulated our motivation towards implementing the state-of-the-art (SOTA) reinforcement learning (RL) method of proximal policy optimization (PPO). The final representation is a well-defined, ready-to-deploy walking, tripedal, soft-legged robot based on PPO-RL in a SOFA environment. Robot navigation performance is a key metric to be considered for measuring the success resolution. Although in the simulated soft robots case, an 82% success rate in reaching a single goal is a groundbreaking output, we pushed the boundaries to further steps by evaluating the progress under assigning a sequence of goals. While trailing the platform steps, outperforming discovery has been observed with an accumulative squared error deviation of 19 mm. The full code is publicly available at href{https://github.com/tarekshohdy/PPO_SOFA_Soft_Legged_Robot.git}{github.com/tarekshohdy/PPO$ extunderscore$SOFA$ extunderscore$Soft$ extunderscore$Legged$ extunderscore$ Robot.git}

Problem

• Developing a soft-legged robot for harsh terrain navigation
• Creating a SOFA-based simulation for soft robot training
• Implementing PPO-RL to optimize tripedal robot walking performance

Innovation

• PPO-RL for soft-legged robot walking control
• Customized SOFA simulation framework setup
• Tripedal robot navigation with 82% success rate