Vision-Language Models for Deployable Social Robot Navigation: Bridging Semantic Reasoning and Low-Level Control

2026-06-27

AI Summary

This work addresses the challenge of integrating semantic understanding with low-level control in social robot navigation, where existing approaches struggle to effectively model human intentions and social norms, and where vision-language models (VLMs) are difficult to safely deploy in real-time systems. The paper proposes a unified framework that, for the first time, decouples VLM-driven social navigation into three modules: semantic reasoning, spatial grounding, and hybrid control. By leveraging intermediate representation learning, the framework reliably couples high-level semantics with low-level planning. Integrated with a dedicated dataset and evaluation platform, this approach systematically constructs a deployable, socially compliant, and socially intelligent navigation system, offering a key technical pathway for applying VLMs in safety-critical scenarios.

Abstract

Social robot navigation (SRN) requires more than geometric path planning; it demands understanding human intentions, social norms, and contextual cues to generate socially compliant behaviors. Although classical navigation methods provide reliable metric planning and collision avoidance, they often lack the semantic reasoning capabilities necessary for operation in complex human-centered environments. Recent advances in Vision-Language Models (VLMs) have opened new opportunities for SRN by enabling high-level VLM understanding, commonsense reasoning, and natural language interaction. However, a fundamental challenge remains: how to integrate VLMs into real-time, safety-critical navigation systems and reliably translate their high-level reasoning into grounded navigation actions. In this survey, we present a unified perspective of VLM-based SRN and organize existing approaches into three interconnected components: high-level VLM reasoning, low-level planning and control, and intermediate mechanisms that bridge reasoning and action. Based on this perspective, we propose a structured roadmap for coupling VLMs with navigation systems, covering semantic reasoning, evaluators, spatial grounding, intermediate representations, and control modules. The roadmap highlights both the strengths of VLMs and the necessity of hybrid architectures for practical deployment. We further review representative datasets and evaluation platforms developed for SRN. Finally, we discuss key open challenges. This survey aims to provide a foundation for building reliable, socially compliant, and deployable VLM-enabled navigation systems.

Problem

• Vision-Language Models
• Social Robot Navigation
• Semantic Reasoning
• Low-Level Control
• Deployable Systems

Innovation

• Vision-Language Models
• Social Robot Navigation
• Semantic Reasoning
• Spatial Grounding
• Hybrid Architecture