MetaphorVU: Towards Metaphorical Video Understanding

2026-05-25

AI Summary

Current multimodal large language models (MLLMs) exhibit limited performance in metaphorical video understanding, struggling to establish cross-domain semantic mappings—a key bottleneck that hinders their higher-order cognitive capabilities and real-world applicability. To address this gap, this work introduces MetaphorVU-Bench, the first benchmark specifically designed for metaphorical video understanding, and proposes MetaphorBoost, a novel inference-time augmentation framework that leverages a metaphor knowledge graph to facilitate cross-domain mapping. Experimental results demonstrate that existing MLLMs significantly underperform human-level understanding on this task, whereas MetaphorBoost consistently enhances performance across multiple state-of-the-art models, thereby validating the efficacy of knowledge-guided semantic mapping for metaphor comprehension in visual contexts.

Abstract

Metaphorical videos are prevalent across various real-world scenarios to convey complex ideas, and understanding them typically requires high-order cognitive capabilities. The lack of systematic studies on metaphorical video understanding not only constrains the real-world applicability of MLLMs but also impedes the thorough assessment of their high-order cognitive capabilities. To bridge this gap, we propose MetaphorVU-Bench, the first systematic and comprehensive benchmark dedicated to metaphorical video understanding. Through experiments, we find current MLLMs struggle with accurate metaphorical video understanding, lagging far behind human level, primarily due to defective cross-domain mapping. Motivated by this finding, we construct a metaphor knowledge graph as mapping augmentation and propose MetaphorBoost, an inference-time enhancement framework achieving consistent performance improvement. Our benchmark, analysis, and method provide useful insights and a foundation for future research on advancing MLLMs.

Problem

• metaphorical video understanding
• MLLMs
• high-order cognitive capabilities
• cross-domain mapping

Innovation

• metaphorical video understanding
• MetaphorVU-Bench
• cross-domain mapping
• metaphor knowledge graph
• MetaphorBoost