LiveEdit: Towards Real-Time Diffusion-Based Streaming Video Editing

2026-06-25

AI Summary

This work addresses the challenges of temporal inconsistency in background and non-edited regions, as well as high latency hindering real-time interaction in streaming video editing. To this end, the authors propose a causal frame-by-frame editing framework that transfers the editing capability of a bidirectional foundation model to an efficient unidirectional streaming editor through a task-oriented three-stage knowledge distillation pipeline. An autoregressive-guided mask caching mechanism is introduced to accelerate inference, complemented by region-aware computation reuse and causal diffusion modeling. Evaluated on a newly constructed streaming video editing benchmark, the method achieves state-of-the-art visual quality while attaining a real-time inference speed of 12.66 FPS, effectively preserving long-term temporal consistency and enabling practical deployment in interactive and augmented reality applications.

Abstract

Streaming video editing has made rapid progress, yet practical deployment is still limited by two core issues: maintaining stable backgrounds and non-edited regions over time, and achieving the low latency required for real-time interactive scenarios. Meanwhile, recent streaming video generation methods are mostly developed for synthesis and cannot be directly applied to editing due to the strict preservation requirement and region-specific control. In this work, we present a novel streaming video editing framework that performs causal, frame-by-frame editing with strong content preservation and real-time responsiveness. Our key design is a three-stage distillation pipeline that progressively transfers editing capability from a powerful bidirectional foundation model to an efficient unidirectional streaming editor, enabling stable long-horizon edits without sacrificing visual fidelity. To further support real-time deployment, we introduce an AR-oriented mask cache that reuses region-related computation across frames, substantially reducing redundant processing and accelerating inference. Finally, we establish a dedicated benchmark for streaming video editing. Extensive evaluations demonstrate that our method achieves state-of-the-art visual quality among streaming baselines while drastically boosting inference speed to 12.66 FPS, making it suitable for interactive and augmented reality applications.

Problem

• streaming video editing
• content preservation
• real-time latency
• temporal stability
• region-specific control

Innovation

• streaming video editing
• causal editing
• distillation pipeline
• mask cache
• real-time inference