SignNav: Leveraging Signage for Semantic Visual Navigation in Large-Scale Indoor Environments

2026-03-17

AI Summary

This work addresses the challenge that existing embodied navigation methods struggle to leverage semantic cues such as signage for decision-making. We introduce SignNav, a novel indoor navigation task that incorporates semantic understanding of signs, and present LSI-Dataset, a large-scale benchmark for this purpose. To tackle SignNav, we propose the Spatio-Temporal Aware Transformer (START), which features a spatial module that maps sign semantics to physical coordinates and a temporal module that models dependencies across historical states. The model is trained in two stages using DAgger. Our approach achieves a success rate (SR) of 80% and a normalized dynamic time warping (NDTW) score of 0.74 on the val-unseen split, and demonstrates effective real-world deployment in map-free physical environments, confirming its practicality and robustness.

Technology Category

• Intelligent Robots: Localization, Mapping, and Navigation
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Agentic search
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs

Abstract

Humans routinely leverage semantic hints provided by signage to navigate to destinations within novel Large-Scale Indoor (LSI) environments, such as hospitals and airport terminals. However, this capability remains underexplored within the field of embodied navigation. This paper introduces a novel embodied navigation task, SignNav, which requires the agent to interpret semantic hint from signage and reason about the subsequent action based on current observation. To facilitate research in this domain, we construct the LSI-Dataset for the training and evaluation of various SignNav agents. Dynamically changing semantic hints and sparse placement of signage in LSI environments present significant challenges to the SignNav task. To address these challenges, we propose the Spatial-Temporal Aware Transformer (START) model for end-to-end decision-making. The spatial-aware module grounds the semantic hint of signage into physical world, while the temporal-aware module captures long-range dependencies between historical states and current observation. Leveraging a two-stage training strategy with Dataset Aggregation (DAgger), our approach achieves state-of-the-art performance, recording an 80% Success Rate (SR) and 0.74 NDTW on val-unseen split. Real-world deployment further demonstrates the practicality of our method in physical environment without pre-built map.

Problem

• semantic visual navigation
• signage
• large-scale indoor environments
• embodied navigation
• SignNav

Innovation

• SignNav
• semantic visual navigation
• Spatial-Temporal Aware Transformer
• LSI-Dataset
• embodied AI