A Graph-Based Reinforcement Learning Approach with Frontier Potential Based Reward for Safe Cluttered Environment Exploration

2025-04-16

AI Summary

Autonomous exploration in cluttered environments faces a fundamental trade-off between safety and efficiency. Method: This paper proposes a reinforcement learning framework integrating a learnable graph-based policy with explicit safety enforcement. It introduces a frontier-potential-field-inspired reward function to encourage active exploration of unknown regions; dynamically couples a graph neural network (GNN) policy with a real-time safety shield to jointly ensure exploration adaptability and rigorous collision avoidance; and enhances the perception-decision closed loop via front-end object detection and information-gain estimation. Results: Evaluated across diverse simulated cluttered environments, the method achieves significant improvements: +18.7% in exploration coverage, −92% in collision rate, and −76% in safety shield activation frequency—demonstrating superior efficiency, safety, and robustness.

Technology Category

• Search and Optimization: Learning to Search
• Natural Language Processing: Safety and Robustness
• Planning, Routing, and Scheduling: Replanning and Plan Repair

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Autonomous exploration of cluttered environments requires efficient exploration strategies that guarantee safety against potential collisions with unknown random obstacles. This paper presents a novel approach combining a graph neural network-based exploration greedy policy with a safety shield to ensure safe navigation goal selection. The network is trained using reinforcement learning and the proximal policy optimization algorithm to maximize exploration efficiency while reducing the safety shield interventions. However, if the policy selects an infeasible action, the safety shield intervenes to choose the best feasible alternative, ensuring system consistency. Moreover, this paper proposes a reward function that includes a potential field based on the agent's proximity to unexplored regions and the expected information gain from reaching them. Overall, the approach investigated in this paper merges the benefits of the adaptability of reinforcement learning-driven exploration policies and the guarantee ensured by explicit safety mechanisms. Extensive evaluations in simulated environments demonstrate that the approach enables efficient and safe exploration in cluttered environments.

Problem

• Ensures safe navigation in cluttered environments with obstacles
• Combines reinforcement learning with safety shields for exploration
• Maximizes exploration efficiency while minimizing safety interventions

Innovation

• Graph neural network-based exploration greedy policy
• Reinforcement learning with proximal policy optimization
• Potential field reward for unexplored regions