ReBind: Multi-Reference Video Editing via Structured Instructions with Explicit Reference Relationships

2026-07-16

AI Summary

This work addresses the challenge of video editing under multiple reference images, where it is difficult to explicitly model the relationship between visual attributes and their sources. To this end, the authors propose ReBind, a novel framework that embeds explicit reference tokens into semantic instructions to construct structured prompts that precisely bind multi-source visual information. ReBind features a two-stage trained, specialized multimodal large language model, ReBind-Instruct, which generates these structured instructions, coupled with a lightweight, adapted text-to-video model, ReBind-Edit, to achieve coherent editing. Experimental results demonstrate that ReBind significantly outperforms existing open-source methods and general-purpose multimodal large language models in both instruction quality and multi-reference video editing performance.

Abstract

Recent diffusion-based video generation models have made significant progress in multi-reference image-conditioned video editing. However, existing methods still struggle to coordinate information from multiple visual sources accurately. We identify a critical deficiency in existing approaches. Existing editing instructions lack explicit reference relationships, and most multimodal large language models (MLLMs) cannot generate them reliably. To address this problem, we propose ReBind, a systematic framework that introduces semantic instructions with embedded reference tokens as the intermediate representation for multi-reference image-conditioned video editing. Our key insight is embedding reference tokens at semantic positions to eliminate ambiguity and establish precise bindings between visual attributes and their sources. We develop ReBind-Instruct, a specialized MLLM that learns to establish explicit bindings between visual attributes and their reference sources through a two-stage progressive scheme for precise reference relationships. We further develop ReBind-Edit, which enables lightweight adaptation of text-to-video models to coordinate multiple references by binding visual attributes to their designated sources. Extensive experiments demonstrate that ReBind substantially outperforms general-purpose MLLMs in instruction quality and achieves state-of-the-art performance among open-source methods on reference image conditioned video editing. Our project webpage: https://rebind-mrv2v.github.io/.

Problem

• multi-reference video editing
• reference relationships
• visual attributes
• instruction ambiguity
• diffusion-based video generation

Innovation

• multi-reference video editing
• explicit reference relationships
• reference tokens
• semantic instructions
• diffusion-based generation