OpenBench: A New Benchmark and Baseline for Semantic Navigation in Smart Logistics

2025-02-13

AI Summary

Addressing three key challenges in intelligent last-mile logistics—strong reliance on high-precision maps, poor generalization of learning-based navigation methods, and the misalignment of existing benchmarks with real-world deployment scenarios—this paper proposes OpenNav, the first lightweight semantic navigation framework tailored for outdoor residential delivery. OpenNav constructs a scalable map representation from OpenStreetMap, integrates large language models (LLMs) for natural-language instruction parsing, and employs vision-language models (VLMs) for global localization and doorplate recognition; it further incorporates classical path planning with real-time map updating. We introduce OpenBench, the first open-source semantic navigation benchmark explicitly designed for realistic delivery environments. Experiments demonstrate substantial improvements in navigation success rate and robustness in both simulation and physical deployments. All code and datasets are publicly released.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Intelligent Robots: Localization, Mapping, and Navigation
• Search and Optimization: Learning to Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Uses of LLMs and GenAI for marketplace design, bidding, and strategic interactions
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

The increasing demand for efficient last-mile delivery in smart logistics underscores the role of autonomous robots in enhancing operational efficiency and reducing costs. Traditional navigation methods, which depend on high-precision maps, are resource-intensive, while learning-based approaches often struggle with generalization in real-world scenarios. To address these challenges, this work proposes the Openstreetmap-enhanced oPen-air sEmantic Navigation (OPEN) system that combines foundation models with classic algorithms for scalable outdoor navigation. The system uses off-the-shelf OpenStreetMap (OSM) for flexible map representation, thereby eliminating the need for extensive pre-mapping efforts. It also employs Large Language Models (LLMs) to comprehend delivery instructions and Vision-Language Models (VLMs) for global localization, map updates, and house number recognition. To compensate the limitations of existing benchmarks that are inadequate for assessing last-mile delivery, this work introduces a new benchmark specifically designed for outdoor navigation in residential areas, reflecting the real-world challenges faced by autonomous delivery systems. Extensive experiments in simulated and real-world environments demonstrate the proposed system's efficacy in enhancing navigation efficiency and reliability. To facilitate further research, our code and benchmark are publicly available.

Problem

• Autonomous robots in smart logistics
• Scalable outdoor navigation challenges
• Limitations of existing navigation benchmarks

Innovation

• OpenStreetMap-enhanced navigation system
• LLMs for delivery instruction comprehension
• VLMs for global localization updates