MapExRL: Human-Inspired Indoor Exploration with Predicted Environment Context and Reinforcement Learning

2025-03-03

AI Summary

This work addresses budget-constrained autonomous exploration in structured indoor environments. Method: We propose a long-horizon frontier selection framework that integrates global semantic map prediction with deep reinforcement learning. For the first time, we jointly model multi-source contextual cues—including prediction uncertainty, sensor coverage, frontier distance, and remaining budget—using discrete frontier points as the action space (replacing continuous motion primitives). Our approach combines an uncertainty-aware map prediction network with a multi-criteria scoring mechanism to enable budget-aware, efficient information gathering. Results: Evaluated on real-world indoor map datasets, our method achieves an 18.8% improvement in coverage over the best prior state-of-the-art, while significantly reducing exploration path length. These results demonstrate superior interpretability, sample efficiency, and practical performance compared to existing approaches.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Search and Optimization: Learning to Search
• Reasoning under Uncertainty: Sequential Decision Making

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Path planning for robotic exploration is challenging, requiring reasoning over unknown spaces and anticipating future observations. Efficient exploration requires selecting budget-constrained paths that maximize information gain. Despite advances in autonomous exploration, existing algorithms still fall short of human performance, particularly in structured environments where predictive cues exist but are underutilized. Guided by insights from our user study, we introduce MapExRL, which improves robot exploration efficiency in structured indoor environments by enabling longer-horizon planning through reinforcement learning (RL) and global map predictions. Unlike many RL-based exploration methods that use motion primitives as the action space, our approach leverages frontiers for more efficient model learning and longer horizon reasoning. Our framework generates global map predictions from the observed map, which our policy utilizes, along with the prediction uncertainty, estimated sensor coverage, frontier distance, and remaining distance budget, to assess the strategic long-term value of frontiers. By leveraging multiple frontier scoring methods and additional context, our policy makes more informed decisions at each stage of the exploration. We evaluate our framework on a real-world indoor map dataset, achieving up to an 18.8% improvement over the strongest state-of-the-art baseline, with even greater gains compared to conventional frontier-based algorithms.

Problem

• Improves robot exploration efficiency in structured indoor environments
• Enables longer-horizon planning using reinforcement learning and map predictions
• Achieves significant performance gains over state-of-the-art exploration algorithms

Innovation

• Reinforcement learning for longer-horizon planning
• Global map predictions enhance exploration efficiency
• Multiple frontier scoring methods improve decision-making