MVPBench: A Multi-Video Perception Evaluation Benchmark for Multi-Modal Video Understanding

2026-03-23

AI Summary

Existing video understanding benchmarks are largely confined to single videos or static images, making them inadequate for evaluating models’ capacity to comprehend complex cross-temporal and cross-view interactions across multiple videos. To address this gap, this work introduces the first comprehensive benchmark for multimodal multi-video perception, encompassing 14 subtasks and 5,000 structured question-answer pairs. The benchmark integrates both existing datasets and newly annotated video content, covering a diverse range of visual scenarios. Experimental results demonstrate that current state-of-the-art multimodal large language models exhibit significant performance degradation when processing multi-video inputs, revealing critical limitations in their ability to perform coordinated understanding across multiple video streams. These findings underscore the necessity and challenge of the proposed benchmark in advancing multi-video reasoning capabilities.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

The rapid progress of Large Language Models (LLMs) has spurred growing interest in Multi-modal LLMs (MLLMs) and motivated the development of benchmarks to evaluate their perceptual and comprehension abilities. Existing benchmarks, however, are limited to static images or single videos, overlooking the complex interactions across multiple videos. To address this gap, we introduce the Multi-Video Perception Evaluation Benchmark (MVPBench), a new benchmark featuring 14 subtasks across diverse visual domains designed to evaluate models on extracting relevant information from video sequences to make informed decisions. MVPBench includes 5K question-answering tests involving 2.7K video clips sourced from existing datasets and manually annotated clips. Extensive evaluations reveal that current models struggle to process multi-video inputs effectively, underscoring substantial limitations in their multi-video comprehension. We anticipate MVPBench will drive advancements in multi-video perception.

Problem

• multi-video perception
• multi-modal video understanding
• evaluation benchmark
• video comprehension
• multi-video interaction

Innovation

• multi-video perception
• multi-modal video understanding
• benchmark
• multi-modal LLMs
• video reasoning