Learning to Crawl: Latent Model-Based Reinforcement Learning for Soft Robotic Adaptive Locomotion

2025-10-07

AI Summary

Soft-bodied crawling robots struggle to achieve adaptive locomotion under model inaccuracies, sensor noise, and unknown gaits. To address this, we propose a model-based reinforcement learning (MB-RL) framework grounded in latent-state dynamical modeling. Our method employs variational inference to learn a low-dimensional latent dynamics model solely from noisy on-board IMU and time-of-flight (ToF) distance measurements, enabling short-horizon motion prediction and end-to-end gait policy optimization. The key innovation lies in the tight integration of implicit dynamic modeling with an actor-critic RL architecture, eliminating reliance on precise physical models or external perception systems. Simulation results demonstrate that the proposed approach robustly generates adaptive crawling gaits under strong sensor noise, significantly enhancing the autonomy and locomotive adaptability of soft robots in uncertain environments.

Technology Category

• Intelligent Robots: State Estimation
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Planning, Routing, and Scheduling: Model-Based Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Soft robotic crawlers are mobile robots that utilize soft body deformability and compliance to achieve locomotion through surface contact. Designing control strategies for such systems is challenging due to model inaccuracies, sensor noise, and the need to discover locomotor gaits. In this work, we present a model-based reinforcement learning (MB-RL) framework in which latent dynamics inferred from onboard sensors serve as a predictive model that guides an actor-critic algorithm to optimize locomotor policies. We evaluate the framework on a minimal crawler model in simulation using inertial measurement units and time-of-flight sensors as observations. The learned latent dynamics enable short-horizon motion prediction while the actor-critic discovers effective locomotor policies. This approach highlights the potential of latent-dynamics MB-RL for enabling embodied soft robotic adaptive locomotion based solely on noisy sensor feedback.

Problem

• Developing control strategies for soft robotic crawlers
• Addressing model inaccuracies and sensor noise in locomotion
• Discovering adaptive locomotor gaits using reinforcement learning

Innovation

• Latent dynamics model guides actor-critic learning
• Onboard sensors provide observations for locomotion control
• Model-based reinforcement learning optimizes soft robot policies