MMLongEmbed: Benchmarking Multimodal Embedding Models in Long-Context Scenarios

2026-06-05

arXiv.org

AI Summary

研究通过引入MMLongEmbed基准，评估多模态嵌入模型在长上下文场景中的表现，揭示现有模型在深度语义和结构依赖性上的不足。

Abstract

Recent advancements have significantly expanded the theoretical context windows of Multimodal Embedding Models (MEMs). However, larger context windows do not necessarily translate into effective comprehension and representation of long-context multimodal inputs, which remains a critical bottleneck for real-world deployment. To address the lack of systematic evaluation in this setting, we introduce MMLongEmbed, the first comprehensive benchmark for evaluating MEMs in long-context scenarios. MMLongEmbed comprises four retrieval tasks spanning multiple context-length ranges, covering text, document, and video modalities. Through extensive evaluation of state-of-the-art models, we find that current architectures rely heavily on superficial feature matching and struggle to capture deep semantic and structural dependencies. We further observe that performance degradation varies systematically with context length and key information placement. Moreover, models exhibit substantially different robustness to redundant contextual information across modalities. For reproducibility, the benchmark and code are publicly available.

Problem

• Multimodal Embedding Models
• long-context scenarios
• comprehensive benchmark
• semantic and structural dependencies
• context length

Innovation

• Multimodal Embedding Models
• Long-Context Scenarios
• Benchmarking
• Semantic and Structural Dependencies
• Redundant Contextual Information