Unlocking the Potential of Grounding DINO in Videos: Parameter-Efficient Adaptation for Limited-Data Spatial-Temporal Localization

2026-04-14

AI Summary

This work addresses the challenges of high annotation costs, data scarcity, and the lack of temporal awareness in existing zero-shot models for video spatio-temporal grounding. To this end, the authors propose ST-GD, a parameter-efficient framework that freezes the backbone of a pretrained vision-language model (e.g., Grounding DINO) and introduces a lightweight spatio-temporal adapter along with a boundary-prediction temporal decoder. With only approximately 10 million trainable parameters, ST-GD enables efficient fine-tuning while preserving pretrained priors and substantially mitigating overfitting under limited supervision. The method achieves state-of-the-art performance on HC-STVG v1 and v2 benchmarks and demonstrates strong generalization capabilities on VidSTG.

Technology Category

• Natural Language Processing: Language Grounding & Multi-modal NLP
• Computer Vision: Large Vision Models
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Spatio-temporal video grounding (STVG) aims to localize queried objects within dynamic video segments. Prevailing fully-trained approaches are notoriously data-hungry. However, gathering large-scale STVG data is exceptionally challenging: dense frame-level bounding boxes and complex temporal language alignments are prohibitively expensive to annotate, especially for specialized video domains. Consequently, conventional models suffer from severe overfitting on these inherently limited datasets, while zero-shot foundational models lack the task-specific temporal awareness needed for precise localization. To resolve this small-data challenge, we introduce ST-GD, a data-efficient framework that adapts pre-trained 2D visual-language models (e.g., Grounding DINO) to video tasks. To avoid destroying pre-trained priors on small datasets, ST-GD keeps the base model frozen and strategically injects lightweight adapters (~10M trainable parameters) to instill spatio-temporal awareness, alongside a novel temporal decoder for boundary prediction. This design naturally counters data scarcity. Consequently, ST-GD excels in data-scarce scenarios, achieving highly competitive performance on the limited-scale HC-STVG v1/v2 benchmarks, while maintaining robust generalization on the VidSTG dataset. This validates ST-GD as a powerful paradigm for complex video understanding under strict small-data constraints.

Problem

• spatio-temporal video grounding
• data scarcity
• video object localization
• limited-data adaptation
• temporal awareness

Innovation

• parameter-efficient adaptation
• spatio-temporal video grounding
• frozen foundation model
• lightweight adapter
• temporal decoder