Mobile Robot Navigation Using Hand-Drawn Maps: A Vision Language Model Approach

2025-01-31

AI Summary

To address navigation failures in robots caused by inaccuracies in hand-drawn maps—such as scale distortion and missing landmarks—this paper proposes a robust autonomous navigation method tailored for hand-drawn maps. The method integrates pre-trained vision-language models (VLMs), topological map representations, and predictive navigation planning. Its core contributions are: (1) a novel Selective Visual Association Prompting mechanism that enables cross-modal topological localization and path planning; and (2) a Predictive Navigation Plan Parser that automatically infers and restores missing landmarks using semantic and spatial priors. Evaluated in both rasterized simulation and real-world user studies, the approach significantly improves navigation success rate and Success weighted by Path Length (SPL). Results demonstrate strong generalization across diverse drawing styles, robot morphologies, and environmental scenarios, as well as cross-platform robustness.

Technology Category

• Intelligent Robots: Localization, Mapping, and Navigation
• Planning, Routing, and Scheduling: Planning with Language Models
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search

Abstract

Hand-drawn maps can be used to convey navigation instructions between humans and robots in a natural and efficient manner. However, these maps can often contain inaccuracies such as scale distortions and missing landmarks which present challenges for mobile robot navigation. This paper introduces a novel Hand-drawn Map Navigation (HAM-Nav) architecture that leverages pre-trained vision language models (VLMs) for robot navigation across diverse environments, hand-drawing styles, and robot embodiments, even in the presence of map inaccuracies. HAM-Nav integrates a unique Selective Visual Association Prompting approach for topological map-based position estimation and navigation planning as well as a Predictive Navigation Plan Parser to infer missing landmarks. Extensive experiments were conducted in photorealistic simulated environments, using both wheeled and legged robots, demonstrating the effectiveness of HAM-Nav in terms of navigation success rates and Success weighted by Path Length. Furthermore, a user study in real-world environments highlighted the practical utility of hand-drawn maps for robot navigation as well as successful navigation outcomes.

Problem

• Robot Navigation
• Hand-drawn Maps
• Map Errors

Innovation

• HAM-Nav
• Adaptive Navigation
• Robotics Mapping