Evaluation of Vision-Language Models Across Diverse Coastal Environments

2026-09-09

AI Summary

研究通过创建一个包含1000多张海岸图像的数据集，并评估七种视觉-语言模型在该环境下的表现，探讨了这些模型在海岸环境中识别物体的挑战及影响因素。

Abstract

Vision-language models (VLMs) enable robotic per- ception by associating visual observations with natural-language concepts. Yet their performance in coastal environments remains largely unexplored. We introduce a densely labeled coastal dataset containing more than 1,000 images collected across seven missions in three regions of Oahu, Hawaii, with 18 semantic classes and over 7,400 annotated instances. We evaluate seven modern VLMs through three complementary experiments mea- suring text-to-mask, mask-to-mask, and mask-to-text alignment. Broad landscape classes are generally recognized more accurately than conventional object and coastal classes, with coastal con- cepts presenting the greatest challenge. However, comparisons of shared conventional classes across coastal and terrestrial datasets reveal no consistent performance difference attributable solely to environmental context. Mask-to-mask matching also remains similar across conventional and coastal classes, while alternative textual labels substantially improve recognition of several coastal concepts. These results suggest that lower performance on coastal classes (at least on the objects/query categories evaluated) is heavily influenced by segmentation and linguistic representation.

Problem

• Vision-Language Models
• Coastal Environments
• Performance Evaluation

Innovation

• coastal dataset
• vision-language models
• text-to-mask
• mask-to-text alignment
• linguistic representation