STRIDE: When to Speak Meets Sequence Denoising for Streaming Video Understanding

2026-03-29

AI Summary

This work addresses the joint decision challenge of “when to respond” and “how to respond” in streaming video understanding by proposing a Structure-aware Prompt Activation Network (SPAN). It introduces, for the first time, a masked diffusion mechanism into response timing prediction. By employing sliding temporal windows to jointly model activation signals across frames and integrating a lightweight masked diffusion module with an iterative denoising strategy, SPAN achieves structured temporal refinement at the activation interface. The method significantly improves both the accuracy and temporal consistency of response timing across multiple streaming video benchmarks and downstream tasks, enabling more reliable and coherent proactive interaction capabilities.

Abstract

Recent progress in video large language models (Video-LLMs) has enabled strong offline reasoning over long and complex videos. However, real-world deployments increasingly require streaming perception and proactive interaction, where video frames arrive online and the system must decide not only what to respond, but also when to respond. In this work, we revisit proactive activation in streaming video as a structured sequence modeling problem, motivated by the observation that temporal transitions in streaming video naturally form span-structured activation patterns. To capture this span-level structure, we model activation signals jointly over a sliding temporal window and update them iteratively as new frames arrive. We propose STRIDE (Structured Temporal Refinement with Iterative DEnoising), which employs a lightweight masked diffusion module at the activation interface to jointly predict and progressively refine activation signals across the window. Extensive experiments on diverse streaming benchmarks and downstream models demonstrate that STRIDE shows more reliable and temporally coherent proactive responses, significantly improving when-to-speak decision quality in online streaming scenarios.

Problem

• streaming video understanding
• when-to-speak
• proactive interaction
• temporal activation
• online perception

Innovation

• structured sequence modeling
• span-structured activation
• masked diffusion
• streaming video understanding
• iterative denoising