Sem-NaVAE: Semantically-Guided Outdoor Mapless Navigation via Generative Trajectory Priors

2026-02-01

AI Summary

This work proposes a semantic-guided real-time navigation method to address the challenges of lacking semantic understanding and efficient path planning in mapless outdoor navigation. The approach uniquely integrates open-vocabulary semantic segmentation with generative trajectory priors: a lightweight vision-language model interprets natural language instructions and performs open-vocabulary semantic segmentation, while a conditional variational autoencoder (CVAE) generates diverse feasible trajectories. The system selects the optimal path in real time based on semantic scoring and executes velocity control via a local planner. Experiments in real-world outdoor environments demonstrate that the method outperforms state-of-the-art approaches in both navigation success rate and trajectory diversity, achieving efficient and semantically aware mapless navigation.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Computer Vision: Language and Vision
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

This work presents a mapless global navigation approach for outdoor applications. It combines the exploratory capacity of conditional variational autoencoders (CVAEs) to generate trajectories and the semantic segmentation capabilities of a lightweight visual language model (VLM) to select the trajectory to execute. Open-vocabulary segmentation is used to score and select the generated trajectories based on natural language, and a state-of-the-art local planner executes velocity commands. One of the key features of the proposed approach is its ability to generate a large variability of trajectories and to select them and navigate in real-time. The approach was validated through real-world outdoor navigation experiments, achieving superior performance compared to state-of-the-art methods. A video showing an experimental run of the system can be found in https://www.youtube.com/watch?v=i3R5ey5O2yk.

Problem

• mapless navigation
• outdoor navigation
• semantic guidance
• trajectory selection
• natural language

Innovation

• semantic navigation
• conditional variational autoencoder
• visual language model
• open-vocabulary segmentation
• mapless navigation