ScanFocus: A Coarse-to-Fine Framework for Spatio-Temporal Video Grounding

2026-07-14

AI Summary

This work addresses the challenge that existing methods struggle to simultaneously model global context and precisely localize temporal boundaries, often resorting to low-frame-rate sampling due to high computational costs and neglecting explicit inter-frame dependencies. To overcome these limitations, we propose a coarse-to-fine two-stage framework: first generating coarse proposals via a unified vision-language encoder coupled with a lightweight semantic-motion fusion module, then densely sampling around proposal boundaries and refining them using a Semantic-Guided Temporal Aggregator (SGTA). Our approach innovatively decouples the task into global scanning and local focusing, and introduces semantic-guided explicit short-range temporal modeling to recover high-frequency boundary cues. Extensive experiments on three mainstream benchmarks demonstrate significant performance gains over state-of-the-art methods, achieving superior spatio-temporal video grounding accuracy.

Abstract

Spatio-Temporal Video Grounding (STVG) aims to retrieve the visual trajectory of a specific object from a video stream as described by a natural language expression. However, most advanced methods struggle to balance global context modeling with precise boundary localization. Due to the prohibitive computational costs of processing long videos, these approaches typically resort to low-rate temporal downsampling and implicit motion modeling. This inevitably suppresses high-frequency boundary cues and neglects the explicit inter-frame dependencies required for precise boundary delineation. To address these limitations, we present \textbf{ScanFocus}, a novel coarse-to-fine framework that decouples the STVG task into a global spatio-temporal scan and a local boundary focus. Specifically, we utilize a unified vision-language fusion encoder combined with a lightweight Deformable Semantic-Motion Fusion module to efficiently align multimodal features and generate coarse proposals. To recover the suppressed fine-grained details, we introduce the Semantic-Guided Temporal Aggregator (SGTA) in the refinement stage. By densely sampling around coarse boundaries, SGTA explicitly models short-term temporal interactions under semantic guidance, capturing rapid motion changes for precise timestamp regression. Extensive experiments on three widely used benchmarks demonstrate the performance superiority of our proposed method over previous approaches. Code will be released at https://github.com/TenMinutes209/ScanFocus.

Problem

• Spatio-Temporal Video Grounding
• boundary localization
• temporal downsampling
• motion modeling
• inter-frame dependencies

Innovation

• coarse-to-fine framework
• spatio-temporal video grounding
• semantic-guided temporal aggregation
• deformable semantic-motion fusion
• boundary localization