Foresee-to-Ground: From Predictive Temporal Perception to Evidence-Driven Reasoning for Video Temporal Grounding

2026-05-21

AI Summary

Existing video large language models often generate timestamps directly from unstructured visual tokens for temporal localization, which can lead to numerical instability and inconsistent boundary predictions. This work proposes reframing the task into a two-stage “identify–measure” framework: first constructing an evidence pool of candidate event segments, then guiding the large language model to predict boundaries based on explicit event hypotheses. The approach innovatively decouples event identification from boundary measurement by introducing referable evidence units and temporally sensitive representations, along with an evidence-driven collaborative reasoning mechanism. Experiments demonstrate that this framework significantly improves localization accuracy across multiple benchmarks, is compatible with diverse video large language model backbones, and preserves general video understanding capabilities.

Abstract

Current Video-LLM approaches for Video Temporal Grounding (VTG) typically rely on direct timestamp generation from an unstructured visual-token stream, often leading to brittle numerics and inconsistent boundaries. To address this, we propose Foresee-to-Ground (F2G), a framework that reformulates VTG as a verifiable Identify-then-Measure problem. F2G integrates Predictive Temporal Perception with Evidence-Driven Reasoning: it learns boundary-sensitive temporal representations to build a video-wide evidence pool of candidate event segments, and exposes these segments to the LLM as citable evidence units that bind boundary prediction to explicit event hypotheses. By decoupling event identification from precise boundary measurement, F2G stabilizes grounding and makes predictions verifiable. Extensive experiments demonstrate that F2G consistently improves grounding accuracy across diverse benchmarks, transfers robustly across different Video-LLM backbones, and preserves general video understanding capabilities.

Problem

• Video Temporal Grounding
• Video-LLM
• timestamp generation
• boundary inconsistency
• temporal perception

Innovation

• Video Temporal Grounding
• Predictive Temporal Perception
• Evidence-Driven Reasoning
• Identify-then-Measure
• Boundary-Sensitive Representation