VisAgent: Narrative-Preserving Story Visualization Framework

2025-03-04

AI Summary

Existing story visualization methods overemphasize visual consistency while neglecting narrative structure and authorial intent, resulting in generated images that fail to accurately convey plot logic and emotional nuance. To address this, we propose a training-free multi-agent framework that enables end-to-end image sequence generation through collaborative agents, jointly optimizing narrative fidelity, semantic consistency, and cross-frame contextual coherence. Our key contributions are: (1) the first narrative-structure-driven hierarchical prompt refinement mechanism, dynamically integrating scene, subject, layout, and other generative elements into a unified workflow; and (2) an integrated pipeline comprising narrative parsing, prompt engineering optimization, diffusion model invocation, and inter-frame semantic alignment. Quantitative and qualitative evaluations demonstrate significant improvements over baselines across multiple narrative fidelity metrics—particularly in visualizing critical plot points and affective details—establishing a novel paradigm for story-driven image generation.

Technology Category

• Humans and AI: Game Design — Procedural Content Generation & Storytelling
• Natural Language Processing: Prompt Engineering / Prompting
• Computer Vision: Diffusion Models for Vision

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Story visualization is the transformation of narrative elements into image sequences. While existing research has primarily focused on visual contextual coherence, the deeper narrative essence of stories often remains overlooked. This limitation hinders the practical application of these approaches, as generated images frequently fail to capture the intended meaning and nuances of the narrative fully. To address these challenges, we propose VisAgent, a training-free multi-agent framework designed to comprehend and visualize pivotal scenes within a given story. By considering story distillation, semantic consistency, and contextual coherence, VisAgent employs an agentic workflow. In this workflow, multiple specialized agents collaborate to: (i) refine layered prompts based on the narrative structure and (ii) seamlessly integrate gt{generated} elements, including refined prompts, scene elements, and subject placement, into the final image. The empirically validated effectiveness confirms the framework's suitability for practical story visualization applications.

Problem

• Transforms narrative elements into coherent image sequences.
• Addresses overlooked narrative essence in story visualization.
• Ensures semantic consistency and contextual coherence in images.

Innovation

• Training-free multi-agent framework for story visualization
• Agentic workflow for narrative comprehension and scene visualization
• Layered prompts and seamless element integration in final images