RefVideo-6M: A Reliable Reference-Based Dataset for Instructional Video Editing

2026-08-26

AI Summary

为解决现有视频编辑数据集存在目标视频质量低及缺乏视觉参考的问题，通过构建包含500万视频样本、100万图像样本及600万视觉参考的RefVideo-6M数据集，并基于此训练了Ref-MoT模型以提高编辑质量与可控性。

Abstract

Recent advances in video editing have been largely driven by large-scale instruction-based datasets. However, existing datasets still suffer from two critical limitations. First, target videos are commonly produced by automatic editing models, which may introduce visible artifacts and unreliable supervision signals. Second, most public datasets rely primarily on textual instructions, while lacking visual references that are crucial for precise, identity-preserving, and controllable editing. To address these limitations, we introduce RefVideo-6M, a large-scale reference-guided editing dataset containing 5 million video editing samples and 1 million image editing samples. To ensure reliable supervision, our dataset uses a construction pipeline that treats artifact-free real videos as editing targets and generates quality-filtered input conditions with multiple editing experts. In addition, it provides approximately 6 million visual references, covering diverse reference types and editing scenarios, thereby enabling models to learn fine-grained visual correspondence beyond text-only instructions. Based on RefVideo-6M, we further train a reference-guided video editing model, Ref-MoT, to evaluate the effectiveness and scalability of the proposed dataset. Extensive experiments demonstrate that RefVideo-6M provides substantially more reliable supervision than existing datasets and enables the training of powerful editing models with improved visual quality, controllability, and reference consistency. The open-source dataset is available at https://huggingface.co/datasets/RefVideo6M/RefVideo6M.

Problem

• video editing
• instruction-based datasets
• artifacts
• visual references

Innovation

• reference-based dataset
• instructional video editing
• visual references
• reliable supervision
• quality-filtered