MAVIN: Multi-Shot Audio-Visual Generation with Narrative Control

2026-06-28

AI Summary

Existing generative models struggle to achieve coherent narrative control in multi-shot audiovisual generation, often suffering from temporal misalignment, character inconsistency, and loosely structured scripts. This work proposes the first multi-shot audiovisual generation framework that supports customizable narrative control, integrating boundary-aware attention, identity-aware propagation modules, and a multi-agent script pipeline to ensure temporal alignment, character consistency, and structured storytelling. The study also introduces MAVINSet, a dedicated dataset, and incorporates hierarchical caption modeling into the audiovisual generation architecture. Experimental results demonstrate that the proposed method significantly outperforms current approaches in temporal coherence, identity stability, and narrative completeness, advancing the applicability of generative models in professional film and television production.

Abstract

While recent generative models produce high-fidelity videos, they struggle with the complex narrative control required for coherent multi-shot audio-visual generation. Existing methods suffer from temporal misalignment, limited controllability, and incomplete scripting. In this paper, we propose MAVIN, the first framework for multi-shot audio-visual generation with customized narrative control. To resolve temporal misalignment, we propose boundary-aware attention, which leverages hierarchical captions and boundary-aware token routing to render audio-visual elements within their respective temporal boundaries. To improve the controllability for multi-subject scenarios, we propose ID-aware propagation, utilizing identity embeddings and an identity-aware mask to bind specific identities to consistent visual appearances and vocal timbres. To provide comprehensive audio-visual narratives, we present a multi-agent scripting pipeline to transform free-form user inputs into hierarchical captions. Furthermore, we construct MAVINSet, a multi-shot audio-visual dataset for robust training and evaluation. Extensive experiments demonstrate that MAVIN achieves state-of-the-art performance, opening up a new avenue for integrating generative models into professional filmmaking workflows.

Problem

• multi-shot audio-visual generation
• narrative control
• temporal misalignment
• controllability
• scripting

Innovation

• multi-shot audio-visual generation
• narrative control
• boundary-aware attention
• ID-aware propagation
• multi-agent scripting