Vision-Language Model Reasoning for Contextual Semantic Mapping in Intralogistics

2026-06-23

AI Summary

This work addresses the limitation of existing autonomous mobile robots in logistics scenarios, which rely solely on geometric maps and lack understanding of object-level semantics and contextual properties such as movability. To overcome this, the authors propose a context-aware semantic mapping approach that integrates SLAM, SAM-based instance segmentation, instance clustering, and multi-view visual language model (VLM) reasoning. Operating in a zero-shot, open-vocabulary setting without task-specific training, the method leverages two prompting strategies to coordinate three VLMs for multi-view semantic fusion. The system achieves a semantic classification mIoU of 98.93% and a movability estimation mean accuracy of 89.17%, substantially enhancing contextual awareness and navigation robustness in dynamic logistics environments.

Abstract

Autonomous mobile robots operating in intralogistics environments rely on geometric maps for localization and navigation, but lack semantic understanding of objects and their contextual properties. We present a contextual semantic mapping pipeline that combines SLAM-based geometric mapping, SAM-based instance segmentation, instance clustering, and VLM multi-view reasoning to produce a contextual semantic map representation encoding geometric structure, object class, and object movability. By aggregating observations across multiple viewpoints and querying a VLM in a zero-shot, open-vocabulary setting, the pipeline infers contextual object properties--here demonstrated through movability--without requiring task-specific training or predefined object categories. We evaluate three VLMs under two prompting strategies and conduct a component-wise analysis of the pipeline. The proposed pipeline achieves 98.93 % mIoU for semantic classification and 89.17 % mAcc for object movability estimation. Component analysis identifies VLM reasoning as the primary bottleneck for contextual understanding and instance clustering as the main limitation for panoptic performance. The resulting semantic map supports context-aware filtering and robust navigation in dynamic intralogistics environments.

Problem

• contextual semantic mapping
• intralogistics
• vision-language model
• object movability
• semantic understanding

Innovation

• Vision-Language Model
• Contextual Semantic Mapping
• Zero-shot Reasoning
• Instance Clustering
• Intralogistics