StreamOPD: A Post-Training Recipe with Spatio-Temporal Cue Gating for Streaming Video Understanding

2026-08-17

AI Summary

This study addresses the substantial post-training inference memory overhead in streaming video understanding by proposing StreamOPD, an efficient optimization framework. The method integrates thought-pattern strategy distillation with a spatiotemporal cue gating mechanism and employs verifiable data reweighting to guide student model learning, achieving performance gains without additional inference costs. Furthermore, a self-distillation extension is introduced to enhance hallucination detection capabilities. Experimental results demonstrate that StreamOPD achieves 83.9% on StreamingBench and yields a 9.1-point improvement on OVO-Bench. Notably, the self-distillation variant outperforms base models across four benchmarks, effectively balancing computational efficiency with accuracy in streaming video comprehension tasks.

Abstract

Streaming video understanding demands direct responses from the causally observed prefix of an unfolding video. Existing systems add inference-time memory, retrieval, and compression, yet a training-free sliding-window baseline already matches them. We therefore fix a memory-free recent-window protocol and ask how far post-training alone can go. Reinforcement learning with verifiable rewards fits this regime poorly, encouraging long ``think-then-answer'' generations, while on-policy distillation (OPD) supplies dense token-level teacher supervision on student trajectories but is stable only when both models train in thinking mode. These observations lead to \textsc{StreamOPD}, a recipe combining verifiable streaming-video data, thinking-mode OPD, and instruct-mode deployment. It raises StreamingBench from $77.9\%$ to $83.9\%$---within $0.3$ points of the 9B teacher---and improves OVO-Bench excluding its hallucination-detection subtask (HLD) by $9.1$ points under unchanged inference. As a teacher-privilege extension, \emph{Spatio-Temporal CueGate (ST-CueGate)} aggregates cue-versus-no-cue teacher likelihood ratios into a group-relative response score that reweights OPD. It reaches $71.9\%$ on OVO-Bench (excluding HLD) and $64.9\%$ on Video-MME, and is the only variant that stays above the base model on all four benchmarks. Replacing the teacher with a frozen copy of the student's initial policy---on-policy self-distillation---retains most of these gains and lifts HLD to $57.0\%$, above both the untrained student and the 9B teacher, so abstention loss is not intrinsic to the recipe. We provide a transparent and reproducible reference for open-source streaming-video research.

Problem

• Streaming Video Understanding
• Post-Training
• On-Policy Distillation
• Memory-Free Protocol

Innovation

• StreamOPD
• On-Policy Distillation
• Spatio-Temporal Cue Gating
• Streaming Video Understanding
• Self-Distillation