CoVStream: Edge-Cloud Collaboration for Understanding of Long Video Streams

2026-06-21

AI Summary

This work addresses the challenge of achieving both low bandwidth overhead and high inference performance when processing long video streams on edge devices. The authors propose the first edge-cloud collaborative framework for long-form video understanding: at the edge, raw video is distilled into compact visual features and semantic captions, which are then transmitted to the cloud; in the cloud, an entity graph and global visual context are constructed, and large models are activated on-demand during query time to perform heavy-weight reasoning. This architecture significantly reduces bandwidth consumption by 87.6% while preserving 99.2% of the accuracy achieved by a full cloud-based baseline on LVBench, thereby enabling efficient and highly accurate long video understanding.

Abstract

Long, continuous video streams are an increasingly critical driver of multimedia intelligence. Existing efforts often handle long videos with a sample-encode-reason approach using large models. However, they overlook a crucial deployment fact: the stream is often produced by computationally constrained devices. This forces an untenable compromise: cloud offloading unlocks strong reasoning but incurs prohibitive bandwidth overhead, while on-device processing remains limited by edge hardware capacity. Therefore, we propose CoVStream, the first edge-cloud collaborative framework for understanding long video streams. The edge node distills raw video streams into compact visual features and semantic captions for transmission to the cloud, minimizing bandwidth costs, while the cloud server integrates this data into an entity graph and global visual context, activating the heavy reasoning model only when a user query arrives. Experiments on VideoMME-Long, LVBench, and RTV-Bench show that CoVStream reduces bandwidth usage by 87.6% while retaining 99.2% of the cloud baseline accuracy on LVBench.

Problem

• long video streams
• edge-cloud collaboration
• bandwidth overhead
• computationally constrained devices
• multimedia intelligence

Innovation

• edge-cloud collaboration
• long video understanding
• bandwidth-efficient streaming
• semantic distillation
• on-demand reasoning