Can Vision-Language Models Assess Proxemic Risk from Egocentric Robot Images?

2026-08-12

AI Summary

This study addresses the challenge of assessing proxemic danger in human environments from a robot’s first-person perspective to enhance embodied navigation safety. We evaluate the performance of three vision-language models—InternVL, Qwen-VL, and SmolVLM—on a four-class proximity risk classification task, systematically comparing multiple prompting strategies and two rounds of QLoRA fine-tuning. We further analyze the relationship between models’ spatial localization capabilities and their risk judgment accuracy. Results show that unmodified models perform near random baseline levels; while overall gains from fine-tuning remain modest, Qwen-VL combined with advanced prompting significantly improves recall in high-risk scenarios. Our findings highlight current limitations of vision-language models in fine-grained proxemic reasoning and spatial grounding, demonstrate that targeted prompting can effectively mitigate model deficiencies, and reveal that correct risk classification does not necessarily rely on accurate spatial attention.

Abstract

Assessing proxemic danger from a robot's egocentric perspective is critical for safe embodied navigation in human environments and requires both visual and contextual reasoning. We evaluate three opensource vision-language models (VLMs) (\textit{InternVL}, \textit{Qwen-VL}, and \textit{SmolVLM}) on the classification of egocentric robot images into four danger levels, comparing three prompting strategies and two rounds of QLoRA fine-tuning against a stratified random baseline. Without fine-tuning, all models perform near the baseline, while fine-tuning yields only modest overall improvements. However, \textit{Qwen-VL} with an advanced prompt achieves substantially higher recall for high-danger cases than the other models. An analysis of person localization further shows that correct danger classification does not correspond to better spatial grounding, indicating that a model may produce a useful safety label without attending to the relevant region of the scene. These results show that current VLMs remain limited in fine-grained proxemic reasoning and spatial grounding, although targeted prompting and fine-tuning can improve high-danger detection in selected models.

Problem

• proxemic risk
• egocentric robot images
• vision-language models
• embodied navigation
• spatial grounding

Innovation

• vision-language models
• proxemic risk
• egocentric robot vision
• spatial grounding
• QLoRA fine-tuning