TAG: A Simple Yet Effective Temporal-Aware Approach for Zero-Shot Video Temporal Grounding

2025-08-11

AI Summary

Existing zero-shot temporal video grounding methods face two key challenges: semantic fragmentation—where semantically coherent frames are erroneously segmented—and distorted similarity distributions—hindering reliable selection of optimal temporal proposals—while often relying on computationally expensive large language models (LLMs). This paper proposes a training-free, time-aware grounding framework. Our method leverages a pre-trained vision-language model and integrates sliding-window scoring with end-to-end inference, eliminating LLM dependency entirely. Key contributions include: (1) temporal pooling to mitigate frame-level semantic discontinuities; (2) temporal consistency clustering to enhance proposal coherence; and (3) similarity self-adaptation to correct distributional bias. Evaluated on Charades-STA and ActivityNet Captions, our approach achieves state-of-the-art performance in both accuracy and efficiency, demonstrating robust zero-shot generalization without parameter tuning or fine-tuning.

Technology Category

• Natural Language Processing: Language Grounding & Multi-modal NLP
• Computer Vision: Video Understanding & Activity Analysis
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Large language models for search

Abstract

Video Temporal Grounding (VTG) aims to extract relevant video segments based on a given natural language query. Recently, zero-shot VTG methods have gained attention by leveraging pretrained vision-language models (VLMs) to localize target moments without additional training. However, existing approaches suffer from semantic fragmentation, where temporally continuous frames sharing the same semantics are split across multiple segments. When segments are fragmented, it becomes difficult to predict an accurate target moment that aligns with the text query. Also, they rely on skewed similarity distributions for localization, making it difficult to select the optimal segment. Furthermore, they heavily depend on the use of LLMs which require expensive inferences. To address these limitations, we propose a extit{TAG}, a simple yet effective Temporal-Aware approach for zero-shot video temporal Grounding, which incorporates temporal pooling, temporal coherence clustering, and similarity adjustment. Our proposed method effectively captures the temporal context of videos and addresses distorted similarity distributions without training. Our approach achieves state-of-the-art results on Charades-STA and ActivityNet Captions benchmark datasets without rely on LLMs. Our code is available at https://github.com/Nuetee/TAG

Problem

• Addresses semantic fragmentation in zero-shot video temporal grounding.
• Corrects skewed similarity distributions for accurate segment localization.
• Eliminates reliance on expensive LLM inferences for efficiency.

Innovation

• Temporal pooling for context capture
• Temporal coherence clustering technique
• Similarity adjustment without LLMs