Closed-Loop Triplet Synergistic Generation for Long-Form Video

2026-06-14

AI Summary

This work addresses the challenges of identity drift and cross-shot inconsistency in multi-shot long-form video generation by proposing a triadic collaborative closed-loop generation paradigm. The approach introduces a vision–language–memory agent framework that jointly models narrative intent, persistent memory, and generated visuals. Central to the method are an entity-centric memory evolution mechanism and a dual-path (intra-shot and inter-shot) dynamic prompt optimization strategy. These components are integrated with a vision–language model analyzer, a mutable visual state memory structure, image-to-video prompt fine-tuning, and evidence-driven prompt rewriting grounded in generation outputs. Evaluated on the StoryBench benchmark, the proposed method significantly outperforms existing state-of-the-art approaches in cross-shot consistency, prompt adherence, and cinematic continuity.

Abstract

Multi-shot long-form video generation remains challenging due to identity drift and compounding inconsistencies across shots. While storyboard-driven pipelines improve controllability, they are often executed in a feed-forward manner, with limited mechanisms to incorporate generated visual evidence back into subsequent conditioning. We propose CoTriSyGen, an agentic framework that formulates multi-shot long video generation as a closed-loop visual-text-memory synergy process, where planned intent, persistent memory, and generated visuals are jointly leveraged for iterative correction and long-range coherence. A vision-language-model-based analyzer reasons over this triplet and produces updates to both prompts and memory along two pathways: (i) intra-shot refinement, which triggers targeted regeneration when semantic or compositional violations are detected and refines image-to-video prompt for coherent motions; and (ii) inter-shot refinement, which rewrites subsequent-shot prompts to propagate newly manifested entities or attributes and improve prompt quality (e.g., compositional grounding and cinematic fluency) based on generated evidence. The loop is grounded in an entity-centric memory modeled as a mutable visual state that evolves as the story progresses, which is continuously updated by both the generator and the analyzer by adding new and evolved entities to reflect appearance changes, accumulated multi-view evidence, and multi-entity compositions. Experiments on our curated StoryBench benchmark demonstrate substantial improvements in cross-shot consistency, prompt adherence, and cinematic continuity over representative methods.

Problem

• long-form video generation
• identity drift
• cross-shot consistency
• multi-shot video
• visual coherence

Innovation

• closed-loop generation
• triplet synergy
• entity-centric memory
• multi-shot video generation
• vision-language reasoning