Vera: A Layered Diffusion Model for Content-Preserving Video Editing

2026-06-22

AI Summary

Existing video editing diffusion models often compromise unmodified original content—such as characters or backgrounds—due to full-frame regeneration. To address this, this work proposes Vera, a hierarchical diffusion framework that architecturally decouples creative editing from content preservation by generating editable layers along with their corresponding alpha mattes and compositing them onto the source video. Built upon an extended text-to-video DiT architecture, Vera introduces a Mixture-of-Transformers design that leverages joint self-attention to enable inter-layer interaction. The method is trained on a high-quality layered video dataset comprising 486,000 frames with precise alpha mattes. Experimental results demonstrate that Vera significantly outperforms state-of-the-art open-source models in content preservation while maintaining competitive editing quality.

Abstract

Video diffusion models have enabled remarkable progress in video generation and editing. However, content preservation remains a core challenge: existing methods regenerate every pixel and often alter elements that should remain unchanged, such as characters or background scenes. We introduce Vera, a layered diffusion framework for content-preserving video editing. Instead of regenerating the entire video, Vera generates an edit layer along with an alpha matte for compositing with the source video, separating creative editing from content preservation by design. To encourage coherent composition with the source video, we extend the text-to-video DiT into a Mixture-of-Transformers (MoT) architecture, with separate DiTs for each layer that interact through joint self-attention. To support the training of Vera, we further construct a high-quality layered dataset with accurate alpha mattes, diverse scenes and dynamics, and visual effects. Across our quantitative benchmark and human preference study, Vera outperforms leading open-source video editing models in content preservation while remaining competitive in edit quality, using 486K frames of layered training data.

Problem

• content preservation
• video editing
• diffusion models
• alpha matte
• layered representation

Innovation

• layered diffusion
• content-preserving video editing
• alpha matte
• Mixture-of-Transformers
• video compositing