AnyGroundBench: A Specialized-Domain Benchmark for Video Grounding in Vision-Language Models

2026-07-02

AI Summary

Existing vision-language models lack effective evaluation benchmarks for spatiotemporal video grounding in specialized domains, making it difficult to assess their ability to adapt to rare concepts and complex dynamics. This work proposes AnyGroundBench—the first video grounding benchmark targeting five specialized domains: animals, industry, sports, surgery, and public safety—integrating newly collected videos with existing datasets to provide high-fidelity dense annotations and a standardized training–testing protocol. Evaluating 15 state-of-the-art models under realistic computational constraints using zero-shot and in-context learning (ICL) frameworks reveals significant performance gaps in professional scenarios, exposing fundamental limitations in complex spatiotemporal reasoning and advocating for a shift toward systematic domain-adaptive evaluation paradigms.

Abstract

Vision-Language Models (VLMs) have demonstrated immense promise in Spatio-Temporal Video Grounding (STVG). However, current evaluation protocols are largely confined to zero-shot assessments on general, daily-life benchmarks. This creates a critical disconnect from real-world applications in specialized fields, where models inevitably encounter rare visual concepts and complex spatio-temporal dynamics. Since exhaustive pre-training across infinite data distributions is infeasible, the ability to adapt to novel domains is essential. To bridge this gap, we introduce AnyGroundBench, a domain-adaptation benchmark designed to shift the STVG evaluation paradigm from static zero-shot testing to rigorous domain adaptation. Targeting five specialized domains (animal, industry, sports, surgery, and public security), AnyGroundBench pairs newly captured videos such as expert-annotated mouse behaviors with established datasets, unifying them through dense, high-fidelity spatio-temporal annotations. Crucially, the benchmark provides dedicated training subsets to systematically measure domain adaptability. We extensively evaluate 15 state-of-the-art VLMs, assessing their zero-shot generalization and In-Context Learning (ICL) capabilities under practical computational constraints. Ultimately, our findings reveal that current models fail in both zero-shot and ICL-based adaptation when confronted with specialized domains, exposing critical flaws in spatio-temporal reasoning that future research must address.

Problem

• Spatio-Temporal Video Grounding
• Vision-Language Models
• Domain Adaptation
• Specialized-Domain Benchmark
• Zero-Shot Evaluation

Innovation

• Video Grounding
• Domain Adaptation
• Vision-Language Models
• Spatio-Temporal Reasoning
• Specialized-Domain Benchmark