3D RL-DWA: A Hybrid Reinforcement Learning and Dynamic Window Approach for Goal-Directed Local Navigation in Multi-DoF Robots

2026-05-12

AI Summary

This work addresses the challenge of goal-directed local navigation for highly deformable micro-robots in complex three-dimensional confined environments using only sparse point cloud observations. The authors propose a hybrid local planner that integrates reinforcement learning with the Dynamic Window Approach (DWA), marking the first such fusion applied to three-dimensional deformable robots. This framework jointly optimizes locomotion trajectories and morphological adaptation to maximize volumetric occupancy while efficiently reaching the target. Evaluated through 1,080 trials in vascular network simulations, the method demonstrates significantly improved deformation adaptability and path completion rates compared to purely reinforcement learning–based or model-driven baselines. It achieves near-perfect task success during training and maintains strong robustness in unseen scenarios.

Abstract

In this paper, we present a novel hybrid approach that combines Reinforcement Learning (RL) with Dynamic Window Approach (DWA) for adaptive 3D local navigation of high-degree-of-freedom robotic systems. Our method leverages sparse point cloud data to dynamically adjust both the motion and the shape of a deformable microrobot, enabling the system to navigate toward a goal in complex, constrained environments while maximizing the occupied volume. We evaluate our framework in a simulated vascular network. Experimental results, based on 1080 trials, indicate that integrating RL with a DWA-based local planner significantly enhances both deformation and navigation capabilities compared to a pure RL and a model-based methods. In particular, the proposed autonomous controller consistently achieves high deformation and near-perfect path completion during training and maintains robust performance in unseen scenarios. These findings highlight the potential of hybrid planning strategies for efficient and adaptive 3D navigation under sparse sensory conditions.

Problem

• 3D local navigation
• multi-DoF robots
• deformable microrobot
• sparse sensory conditions
• goal-directed navigation

Innovation

• Reinforcement Learning
• Dynamic Window Approach
• 3D Local Navigation
• Deformable Microrobot
• Sparse Point Cloud