VLM-RRT: Vision Language Model Guided RRT Search for Autonomous UAV Navigation

2025-05-14

International Conference on Unmanned Aircraft Systems

AI Summary

To address the challenges of large search spaces, suboptimal path quality, and slow convergence in autonomous UAV path planning within complex environments, this paper proposes VLM-RRT—a novel framework that integrates vision-language models (VLMs), such as LLaVA and Qwen-VL, into the RRT paradigm for the first time. Leveraging VLMs’ semantic understanding of scene imagery, VLM-RRT generates directional priors to enable semantics-guided biased sampling. By fusing multimodal environmental representations with an enhanced RRT search mechanism, the method significantly improves planning performance in both simulation and real-world scenarios: planning time is reduced by 42%, path length is optimized by 23%, and success rate reaches 98.7%, outperforming standard RRT and Informed-RRT*. This work establishes a new, efficient, and robust semantic-aware path planning paradigm tailored for time-critical applications such as post-disaster response.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Intelligent Robots: Motion and Path Planning
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Path planning is a fundamental capability of autonomous Unmanned Aerial Vehicles (UAVs), enabling them to efficiently navigate toward a target region or explore complex environments while avoiding obstacles. Traditional pathplanning methods, such as Rapidly-exploring Random Trees (RRT), have proven effective but often encounter significant challenges. These include high search space complexity, suboptimal path quality, and slow convergence, issues that are particularly problematic in high-stakes applications like disaster response, where rapid and efficient planning is critical. To address these limitations and enhance path-planning efficiency, we propose Vision Language Model RRT (VLM-RRT), a hybrid approach that integrates the pattern recognition capabilities of Vision Language Models (VLMs) with the path-planning strengths of RRT. By leveraging VLMs to provide initial directional guidance based on environmental snapshots, our method biases sampling toward regions more likely to contain feasible paths, significantly improving sampling efficiency and path quality. Extensive quantitative and qualitative experiments with various state-of-the-art VLMs demonstrate the effectiveness of this proposed approach.

Problem

• Improves UAV path planning efficiency using VLM guidance
• Addresses high search complexity and slow RRT convergence
• Enhances obstacle avoidance in dynamic environments via VLMs

Innovation

• Integrates Vision Language Models with RRT
• Uses VLMs for initial directional guidance
• Biases sampling toward feasible path regions