SportsGrounder: Proposal-Aided Interleaved Grounding for Dense Sports Video Reasoning

2026-08-08

AI Summary

This work addresses the challenge of fine-grained understanding in dense sports videos, where small-scale, visually similar, and highly interactive entities—such as players and balls—often cause existing multimodal large models to be misled by textual priors. To mitigate this, the authors propose a framework that integrates open-vocabulary visual experts with grid features, incorporating domain-guided object proposals, an Interleaved Grounding Fusion (IGF) mechanism, Action-Aware Supervision (AAS), and Mixed Preference Optimization (MPO). The IGF module preserves temporal alignment while alleviating sequence explosion, AAS enhances motion representation learning, and MPO improves robustness against interference. Evaluated on a dense video question-answering benchmark built from SoccerNet and FineSports, the proposed method achieves state-of-the-art accuracy, significantly advancing fine-grained reasoning performance.

Abstract

Sports video analysis is crucial for athletic analytics and broadcasting enhancement. Dense sports video reasoning, however, demands a fine-grained understanding of numerous small-scale, highly interactive, and visually homogeneous entities (e.g., players sharing identical uniforms, the ball) across long temporal contexts. Current Large Multimodal Models (LMMs) inherently struggle with such dense visual complexities. Due to the lack of fine-grained visual details, these models often over-rely on textual priors to guess answers, especially when distinguishing visually similar actions and players. To address this, we propose \textbf{SportsGrounder}, a framework that leverages an open-vocabulary visual expert to aid interleaved grounding specifically for dense sports video reasoning. To achieve precise spatial localization, we extract domain-guided object proposals and introduce an Interleaved Grounding Fusion (IGF) mechanism. The IGF frame-by-frame integrates explicit bounding box coordinates and implicit visual semantics with global grid features. This design preserves strict temporal alignment and prevents sequence length explosion. Furthermore, we design an Action-Aware Supervision (AAS) module that directly regularizes the model's hidden states, forcing the network to learn accurate motion representations rather than relying on language bias. Optimized with Mixed Preference Optimization (MPO) to better distinguish deceptive distractors, our extensive experiments on newly curated dense sports VQA datasets (derived from SoccerNet and FineSports) demonstrate that SportsGrounder significantly improves fine-grained reasoning and achieves state-of-the-art accuracy.

Problem

• dense sports video reasoning
• fine-grained visual understanding
• visually homogeneous entities
• temporal context
• language bias

Innovation

• Interleaved Grounding Fusion
• Action-Aware Supervision
• Domain-Guided Object Proposals
• Mixed Preference Optimization
• Dense Sports Video Reasoning