SignScene: Visual Sign Grounding for Mapless Navigation
Application Category
📝 Abstract
Navigational signs enable humans to navigate unfamiliar environments without maps. This work studies how robots can similarly exploit signs for mapless navigation in the open world. A central challenge lies in interpreting signs: real-world signs are diverse and complex, and their abstract semantic contents need to be grounded in the local 3D scene. We formalize this as sign grounding, the problem of mapping semantic instructions on signs to corresponding scene elements and navigational actions. Recent Vision-Language Models (VLMs) offer the semantic common-sense and reasoning capabilities required for this task, but are sensitive to how spatial information is represented. We propose SignScene, a sign-centric spatial-semantic representation that captures navigation-relevant scene elements and sign information, and presents them to VLMs in a form conducive to effective reasoning. We evaluate our grounding approach on a dataset of 114 queries collected across nine diverse environment types, achieving 88% grounding accuracy and significantly outperforming baselines. Finally, we demonstrate that it enables real-world mapless navigation on a Spot robot using only signs.
Problem
Research questions and friction points this paper is trying to address.
sign grounding
mapless navigation
visual sign interpretation
semantic grounding
robot navigation
Innovation
Methods, ideas, or system contributions that make the work stand out.
sign grounding
mapless navigation
Vision-Language Models
spatial-semantic representation
visual sign interpretation
🔎 Similar Papers
No similar papers found.
