EcoVideo: Entropy-Orchestrated Video Generation Paradigm in Cloud-Edge Dynamics

2026-06-29

AI Summary

This work addresses the high latency in DiT-based video generation caused by full-frame iterative denoising and the limitations of existing cloud-edge collaborative approaches that rely on static frame decoupling, failing to exploit inter-frame similarity or adapt to dynamic system conditions. The authors propose EcoVideo, a novel framework that introduces, for the first time, a training-agnostic, self-attention entropy-driven mechanism to dynamically identify high-information-density keyframes for processing by powerful cloud models, while reconstructing remaining frames via motion-aware interpolation using lightweight edge models. Crucially, EcoVideo adaptively adjusts both the number of keyframes and their refinement depth based on real-time bandwidth and computational constraints. This enables dynamic inter-frame decoupling and real-time optimization under resource limitations, achieving up to 2.9× end-to-end speedup in low-bandwidth, compute-constrained scenarios while significantly improving the quality-efficiency trade-off.

Abstract

DiT video generation is latency-intensive due to iterative full-frame denoising, while prior cloud-edge methods largely rely on static inter-step decoupling and cannot leverage inter-frame similarity or adapt to system dynamics. We propose EcoVideo, an entropy-orchestrated framework for dynamic inter-frame decoupling: early-stage self-attention entropy provides a training-free estimate of frame-wise information density for frame selection; a cloud large model denoises sparse high-entropy keyframes; and an edge lightweight model reconstructs the remaining frames via motion-aware interpolation with refinement for temporal stability. EcoVideo further adapts the keyframe budget and edge refinement depth to real-time bandwidth and compute availability, optimizing end-to-end latency under constraints. Experiments on representative DiT video generators show improved quality--efficiency trade-offs and up to 2.9x end-to-end speedup in low-bandwidth, compute-limited edge settings. Code is available at https://github.com/IF-LAB-PKU/EcoVideo.

Problem

• video generation
• cloud-edge computing
• latency optimization
• frame similarity
• system dynamics

Innovation

• entropy-orchestrated
• cloud-edge collaboration
• dynamic frame decoupling
• motion-aware interpolation
• adaptive resource allocation