From Woofs to Words: Towards Intelligent Robotic Guide Dogs with Verbal Communication

2026-03-12

AI Summary

This work addresses the need for enhanced semantic understanding and spatial awareness in robotic guide dogs operating in dynamic environments for visually impaired users. It proposes a novel dialogue system that integrates a large language model (LLM) to unify natural language generation with real-time environmental perception and navigation planning—a first in this domain. The system translates scene context and path-planning decisions into interpretable natural language, enabling collaborative human–robot decision-making. Through user studies and simulation experiments, the study validates the effectiveness of diverse linguistic expression strategies, demonstrating significant improvements in both interaction efficiency and navigation accuracy.

Technology Category

• Natural Language Processing: Conversational AI/Dialog Systems
• Intelligent Robots: Human-Robot Interaction
• Computer Vision: Language and Vision

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Assisted, interactive, and conversational search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Assistive robotics is an important subarea of robotics that focuses on the well-being of people with disabilities. A robotic guide dog is an assistive quadruped robot that helps visually impaired people in obstacle avoidance and navigation. Enabling language capabilities for robotic guide dogs goes beyond naively adding an existing dialog system onto a mobile robot. The novel challenges include grounding language in the dynamically changing environment and improving spatial awareness for the human handler. To address those challenges, we develop a novel dialog system for robotic guide dogs that uses LLMs to verbalize both navigational plans and scenes. The goal is to enable verbal communication for collaborative decision-making within the handler-robot team. In experiments, we conducted a human study to evaluate different verbalization strategies and a simulation study to assess the efficiency and accuracy in navigation tasks.

Problem

• robotic guide dogs
• verbal communication
• language grounding
• spatial awareness
• assistive robotics

Innovation

• robotic guide dog
• verbal communication
• language grounding
• large language models
• assistive robotics