Beyond the Single Camera: Agentic Multi-View Reasoning in Sports Video Understanding

2026-07-13

AI Summary

Current vision-language models struggle to interpret sports videos due to challenges posed by occlusion, rapid motion, and complex interactions, and there is a lack of multi-view evaluation benchmarks. To address this, this work introduces SportMV-Bench, the first high-quality benchmark for multi-view sports video understanding, along with SportMV-Agent, an agent framework that enhances comprehension through active view selection, perception tool invocation, and evidence-driven iterative reasoning. Experimental results demonstrate that SportMV-Agent achieves a 14.46% relative performance improvement over the strongest baseline on this benchmark, significantly advancing the effective utilization of multi-view information and revealing critical bottlenecks in current models regarding fine-grained perception and view-selection decision-making.

Abstract

Recent Multimodal Large Language Models (MLLMs) achieve strong performance on single-view video understanding benchmarks. However, sports videos involve dense occlusion, rapid motion, and complex interactions that are difficult to resolve from a single viewpoint. In practice, sports events are recorded from multiple camera angles, providing complementary evidence used by referees. Yet, no existing benchmark evaluates MLLMs on multi-view sports video understanding. To address this gap, we introduce SportMV-Bench, a comprehensive benchmark built from official match recordings, through a dedicated pipeline combining LLM-based generation, MLLM-based verification, and human filtering to ensure quality and consistency. SportMV-Bench containing 787 multi-view video bundles and 2592 question-answer pairs across three categories: Perception-Aware Recognition (PAR), Rule-aware Event Interpretation (REI), and Adjudicative Decision Reasoning(ADR). Our analysis shows that current MLLMs fail to effectively exploit multi-view information, with the bottlenecks lying in fine-grained visual perception and view selection rather than logical reasoning or domain knowledge. We propose SportMV-Agent, an agentic framework that orchestrates an iterative loop of active view selection, perception tool execution, and evidence-grounded reasoning, achieving a significant 14.46% relative improvement over the strongest MLLM baseline.

Problem

• multi-view reasoning
• sports video understanding
• multimodal large language models
• occlusion
• camera angles

Innovation

• multi-view reasoning
• sports video understanding
• agentic framework
• perception-aware recognition
• view selection