FH-DRL: Exponential-Hyperbolic Frontier Heuristics with DRL for accelerated Exploration in Unknown Environments

2024-07-26

AI Summary

To address the challenges of autonomous robot exploration and unreliable navigation in unknown, large-scale, and cluttered environments, this paper proposes a collaborative exploration framework integrating an exponential-hyperbolic frontier heuristic with TD3-based deep reinforcement learning. We innovatively design an exponential-hyperbolic distance scoring mechanism, coupled with a stochastic openness metric derived from real-time occupancy estimation, to dynamically balance local reachability and global information gain—thereby significantly improving frontier quality and policy adaptability. Extensive evaluations in simulation and on physical TurtleBot3 platforms demonstrate that our method reduces exploration completion time by 21.3% and path length by 18.7% compared to baseline approaches including Nearest Frontier and CogNet. Moreover, it exhibits markedly enhanced robustness in structured, complex environments such as corridors and mazes.

Technology Category

• Intelligent Robots: State Estimation
• Search and Optimization: Heuristic Search
• Planning, Routing, and Scheduling: Replanning and Plan Repair

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

Autonomous robot exploration in large-scale or cluttered environments remains a central challenge in intelligent vehicle applications, where partial or absent prior maps constrain reliable navigation. This paper introduces FH-DRL, a novel framework that integrates a customizable heuristic function for frontier detection with a Twin Delayed DDPG (TD3) agent for continuous, high-speed local navigation. The proposed heuristic relies on an exponential-hyperbolic distance score, which balances immediate proximity against long-range exploration gains, and an occupancy-based stochastic measure, accounting for environmental openness and obstacle densities in real time. By ranking frontiers using these adaptive metrics, FH-DRL targets highly informative yet tractable waypoints, thereby minimizing redundant paths and total exploration time. We thoroughly evaluate FH-DRL across multiple simulated and real-world scenarios, demonstrating clear improvements in travel distance and completion time over frontier-only or purely DRL-based exploration. In structured corridor layouts and maze-like topologies, our architecture consistently outperforms standard methods such as Nearest Frontier, Cognet Frontier Exploration, and Goal Driven Autonomous Exploration. Real-world tests with a Turtlebot3 platform further confirm robust adaptation to previously unseen or cluttered indoor spaces. The results highlight FH-DRL as an efficient and generalizable approach for frontier-based exploration in large or partially known environments, offering a promising direction for various autonomous driving, industrial, and service robotics tasks.

Problem

• Autonomous robot exploration in large-scale environments
• Integration of heuristic function with TD3 agent
• Minimizing redundant paths and exploration time

Innovation

• Exponential-hyperbolic distance score
• Twin Delayed DDPG agent
• Occupancy-based stochastic measure