StreamFlow: Dynamic Memory Flows for Streaming Video Understanding

2026-08-11

AI Summary

This work addresses the challenge that existing multimodal large language models struggle to efficiently retain critical visual evidence from streaming video under strict causality and limited memory constraints. The authors propose a dynamic visual memory framework that employs a lightweight mid-term memory module to filter temporal redundancy and constructs a latent-space long-term memory to enable on-demand retrieval of historical visual information. An attention-guided mechanism dynamically injects relevant visual latent variables during generation. This approach achieves, for the first time, non-intrusive and dynamic on-demand access to past visual content without modifying the backbone model, thereby overcoming the redundant computation and rigid access patterns of conventional memory mechanisms. Evaluated on StreamingBench, the method attains 67.73% accuracy, improves visual attention scores by 59.1%, and reduces end-to-end latency and peak memory usage by 50.4% and 21.1%, respectively.

Abstract

Streaming video understanding requires multimodal large language models (MLLMs) to preserve relevant evidence from continuously evolving streams under strict causality and bounded memory. Yet existing paradigms remain limited: model-based methods require intrusive backbone updates, while memory-based methods expend substantial visual-encoding computation on temporally redundant content and rely on rigid access to visual history. To address these limitations, we introduce StreamFlow, an efficient visual memory framework that enables dynamic, on-demand access to historical visual information. StreamFlow combines a lightweight, dynamics-aware mid-term memory that filters temporal redundancy before visual encoding with a latent long-term memory that consolidates historical video content into visual latents accessible to subsequent reasoning. During generation, an attention-guided retrieval mechanism injects relevant visual latents when the model's reliance on visual evidence weakens. StreamFlow achieves state-of-the-art streaming video understanding performance, reaching 67.73% overall accuracy on StreamingBench, while also delivering strong performance on offline long-video benchmarks. Relative to the vanilla setting, it improves the visual attention score (VAS) by 59.1% while reducing end-to-end latency and peak memory by 50.4% and 21.1%, respectively, enabling more visually grounded and efficient reasoning.

Problem

• streaming video understanding
• multimodal large language models
• visual memory
• temporal redundancy
• causality

Innovation

• StreamFlow
• dynamic memory
• streaming video understanding
• visual latents
• attention-guided retrieval