ID-Composer: Multi-Subject Video Synthesis with Hierarchical Identity Preservation

2025-11-01

AI Summary

This work addresses key challenges in multi-subject video generation—weak identity consistency, cross-modal semantic misalignment, and poor temporal coherence—by proposing the first framework jointly driven by text prompts and multiple reference images. Methodologically, it introduces a hierarchical identity-preserving attention mechanism to explicitly model multi-subject identity features; leverages a pre-trained vision-language model (VLM) for fine-grained cross-modal semantic alignment; and integrates diffusion modeling with online reinforcement learning to dynamically optimize identity fidelity and temporal consistency. Extensive experiments across multiple benchmarks demonstrate that our approach significantly outperforms existing methods in subject identity preservation, semantic accuracy, and motion coherence. It substantially enhances controllability and visual realism in multi-subject video generation, establishing new state-of-the-art performance.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Natural Language Processing: Generation

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Video generative models pretrained on large-scale datasets can produce high-quality videos, but are often conditioned on text or a single image, limiting controllability and applicability. We introduce ID-Composer, a novel framework that addresses this gap by tackling multi-subject video generation from a text prompt and reference images. This task is challenging as it requires preserving subject identities, integrating semantics across subjects and modalities, and maintaining temporal consistency. To faithfully preserve the subject consistency and textual information in synthesized videos, ID-Composer designs a extbf{hierarchical identity-preserving attention mechanism}, which effectively aggregates features within and across subjects and modalities. To effectively allow for the semantic following of user intention, we introduce extbf{semantic understanding via pretrained vision-language model (VLM)}, leveraging VLM's superior semantic understanding to provide fine-grained guidance and capture complex interactions between multiple subjects. Considering that standard diffusion loss often fails in aligning the critical concepts like subject ID, we employ an extbf{online reinforcement learning phase} to drive the overall training objective of ID-Composer into RLVR. Extensive experiments demonstrate that our model surpasses existing methods in identity preservation, temporal consistency, and video quality.

Problem

• Generating multi-subject videos while preserving distinct identities from references
• Integrating semantic understanding across text prompts and multiple subject images
• Maintaining temporal consistency and identity alignment in synthesized videos

Innovation

• Hierarchical identity-preserving attention mechanism for feature aggregation
• Semantic understanding via pretrained vision-language model guidance
• Online reinforcement learning phase for training objective alignment