RUMOR: Reinforcement learning for Understanding a Model of the Real World for Navigation in Dynamic Environments

2024-04-25

Robotics and Autonomous Systems

AI Summary

To address key challenges in autonomous robot navigation within dynamic environments—including poor generalization, inadequate modeling of kinematic and dynamic constraints, and a substantial simulation-to-reality gap—this paper proposes a robust deep reinforcement learning planner tailored for wheeled robots. Our method explicitly encodes kinematic and dynamic constraints into the action space via a novel robot-centric velocity-space formulation. We further develop a high-fidelity dynamic simulation environment that accurately reproduces real-world complexities such as dense pedestrian flows and previously unseen obstacles. The planner is trained end-to-end to achieve strong cross-scenario generalization. Experiments demonstrate significant performance gains over state-of-the-art methods in densely populated dynamic environments and unknown obstacle configurations. Real-robot deployment validates both its robustness and practical transferability, confirming effective bridging of the simulation-to-reality gap.

Technology Category

• Intelligent Robots: Motion and Path Planning
• Planning, Routing, and Scheduling: Replanning and Plan Repair
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Autonomous navigation in dynamic environments is a complex but essential task for autonomous robots, with recent deep reinforcement learning approaches showing promising results. However, the complexity of the real world makes it infeasible to train agents in every possible scenario configuration. Moreover, existing methods typically overlook factors such as robot kinodynamic constraints, or assume perfect knowledge of the environment. In this work, we present RUMOR, a novel planner for differential-drive robots that uses deep reinforcement learning to navigate in highly dynamic environments. Unlike other end-to-end DRL planners, it uses a descriptive robocentric velocity space model to extract the dynamic environment information, enhancing training effectiveness and scenario interpretation. Additionally, we propose an action space that inherently considers robot kinodynamics and train it in a simulator that reproduces the real world problematic aspects, reducing the gap between the reality and simulation. We extensively compare RUMOR with other state-of-the-art approaches, demonstrating a better performance, and provide a detailed analysis of the results. Finally, we validate RUMOR's performance in real-world settings by deploying it on a ground robot. Our experiments, conducted in crowded scenarios and unseen environments, confirm the algorithm's robustness and transferability.

Problem

• Navigating dynamic environments with robot kinodynamic constraints
• Training agents without perfect environment knowledge
• Bridging the gap between simulation and real-world performance

Innovation

• Uses robocentric velocity space model
• Incorporates robot kinodynamic constraints
• Trains in realistic simulation environments