Stand-In: A Lightweight and Plug-and-Play Identity Control for Video Generation

2025-08-11

AI Summary

This work addresses the problem of high-fidelity, lightweight, user-specified identity control in video generation. Methodologically, we propose a plug-and-play fine-tuning framework that introduces a conditional image branch and a constrained self-attention mechanism, coupled with conditional positional mapping to achieve precise identity preservation. The approach fine-tunes only ~1% of parameters from a pre-trained video diffusion model and requires merely 2,000 paired samples for adaptation. It is compatible with mainstream AIGC tools and supports diverse tasks—including character-driven generation, pose reference, stylization, and face swapping. Experimental results demonstrate that our method significantly outperforms full-parameter fine-tuning baselines in both identity consistency and visual quality, while offering superior efficiency, generalizability, and deployment friendliness.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Generation

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Search and Retrieval-Augmented AI: Agentic search

Abstract

Generating high-fidelity human videos that match user-specified identities is important yet challenging in the field of generative AI. Existing methods often rely on an excessive number of training parameters and lack compatibility with other AIGC tools. In this paper, we propose Stand-In, a lightweight and plug-and-play framework for identity preservation in video generation. Specifically, we introduce a conditional image branch into the pre-trained video generation model. Identity control is achieved through restricted self-attentions with conditional position mapping, and can be learned quickly with only 2000 pairs. Despite incorporating and training just $sim$1% additional parameters, our framework achieves excellent results in video quality and identity preservation, outperforming other full-parameter training methods. Moreover, our framework can be seamlessly integrated for other tasks, such as subject-driven video generation, pose-referenced video generation, stylization, and face swapping.

Problem

• Achieve identity control in video generation with minimal parameters
• Ensure compatibility with existing AIGC tools for seamless integration
• Maintain high video quality while preserving user-specified identities

Innovation

• Lightweight plug-and-play identity control framework
• Conditional image branch with restricted self-attentions
• Minimal training data and parameters for integration