LaViSA: A Language and Vision Structural Ambiguity Benchmark

2026-06-17

AI Summary

Structurally ambiguous sentences, which admit multiple syntactic parses, pose significant challenges for accurate interpretation and often require visual context for disambiguation. This work introduces LaViSA, the first vision–language benchmark specifically designed for structural ambiguity, encompassing seven canonical ambiguity types and providing paired ambiguous and disambiguated sentences along with corresponding images. The dataset enables fine-grained evaluation of vision–language models’ capacity to leverage visual cues for resolving linguistic structural ambiguities. Experimental results demonstrate that while current models can partially exploit visual information to resolve such ambiguities, they still exhibit notable limitations—particularly in handling specific ambiguity types and subtle semantic distinctions.

Abstract

Structural ambiguity arises when a single sentence admits multiple valid interpretations due to its syntactic structure, posing a fundamental challenge for language understanding. Visual scenes serve as useful cues for resolving such ambiguity, and Vision and Language Models (VLMs) need to be capable of deriving possible semantic interpretations from visual scenes. We introduce Language and Vision Structural Ambiguity (LaViSA), a benchmark designed to evaluate the ability of VLMs to resolve structural ambiguity leveraging visual scenes. LaViSA consists of ambiguous sentences, their disambiguated sentences, and corresponding images of these disambiguated sentences across seven ambiguity categories. Using LaViSA, we conduct a comprehensive evaluation of diverse VLMs, including both proprietary and open-source models with varying parameter scales and reasoning capabilities. Experimental results show that although recent VLMs can leverage visual scenes to resolve structural ambiguity to a some extent, they still struggle with certain ambiguity types and visually subtle semantic distinctions, indicating remaining limitations in resolving structural ambiguity using visual scenes.

Problem

• structural ambiguity
• vision and language models
• visual disambiguation
• language understanding
• multimodal reasoning

Innovation

• structural ambiguity
• vision-language models
• multimodal benchmark
• disambiguation
• visual semantics