MuSS: A Large-Scale Dataset and Cinematic Narrative Benchmark for Multi-Shot Subject-to-Video Generation

2026-04-26

AI Summary

This work addresses the challenge that existing video generation models struggle to produce multi-shot sequences aligned with authentic cinematic narratives, primarily due to the scarcity of data that jointly captures narrative logic, spatiotemporal alignment, and subject consistency. To overcome this, the authors introduce MuSS, a large-scale dual-track movie dataset derived from over 3,000 films, accompanied by a progressive annotation pipeline and a cross-shot subject matching mechanism to ensure both local accuracy and global coherence. They further propose a cinematic narrative benchmark and a novel evaluation metric, Anti-Copy-Paste Variance (ACP-Var), to assess narrative continuity and 3D structural consistency. Experiments demonstrate that models trained on MuSS significantly outperform current approaches in narrative coherence and cross-shot identity preservation, effectively avoiding degeneration into simplistic 2D “sticker” generators and marking a substantial advance in multi-shot video generation.

Technology Category

• Computer Vision: Multi-modal Vision
• Natural Language Processing: Generation
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

While video foundation models excel at single-shot generation, real-world cinematic storytelling inherently relies on complex multi-shot sequencing. Further progress is constrained by the absence of datasets that address three core challenges: authentic narrative logic, spatiotemporal text-video alignment conflicts, and the "copy-paste" dilemma prevalent in Subject-to-Video (S2V) generation. To bridge this gap, we introduce MuSS, a large-scale, dual-track dataset tailored for multi-shot video and S2V generation. Sourced from over 3,000 movies, MuSS explicitly supports both complex montage transitions and subject-centric narratives. To construct this dataset, we pioneer a progressive captioning pipeline that eliminates contextual conflicts by ensuring local shot-level accuracy before enforcing global narrative coherence. Crucially, we implement a cross-shot matching mechanism to fundamentally eradicate the S2V copy-paste shortcut. Alongside the dataset, we propose the Cinematic Narrative Benchmark, featuring a visual-logic-driven paradigm and a novel Anti-Copy-Paste Variance (ACP-Var) metric to rigorously assess continuous storytelling and 3D structural consistency. Extensive experiments demonstrate that while current baselines struggle with continuous narrative logic or degenerate into trivial 2D sticker generators, our MuSS-augmented model achieves state-of-the-art narrative effectiveness and cross-shot identity preservation.

Problem

• multi-shot video generation
• cinematic narrative
• Subject-to-Video
• text-video alignment
• copy-paste dilemma

Innovation

• multi-shot video generation
• Subject-to-Video (S2V)
• progressive captioning pipeline
• cross-shot matching
• Anti-Copy-Paste Variance (ACP-Var)