EmoStory: Emotion-Aware Story Generation

2026-03-11

AI Summary

This work addresses the common oversight in existing visual storytelling methods regarding the critical role of emotion in shaping both narrative structure and visual expression, particularly the lack of explicit emotional guidance and cross-frame affective consistency. To bridge this gap, we propose a novel emotion-aware paradigm for visual story generation, featuring a two-stage framework: first, an emotion agent collaborates with a writing agent to produce emotionally coherent story prompts; second, a region-aware generation mechanism ensures subject continuity while integrating emotion-relevant visual elements. We introduce a new dataset comprising 25 subjects and 600 emotion-annotated stories to support this approach. Extensive evaluation—including quantitative metrics, qualitative analysis, and user studies—demonstrates significant improvements over state-of-the-art methods in emotional accuracy, prompt alignment, and subject consistency.

Technology Category

• Humans and AI: Game Design — Procedural Content Generation & Storytelling
• Computer Vision: Visual Reasoning & Symbolic Representations
• Natural Language Processing: Generation

Application Category

• Web Mining and Content Analysis: Web data visualization
• Responsible Web: Consent frameworks and practices on the web
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems

Abstract

Story generation aims to produce image sequences that depict coherent narratives while maintaining subject consistency across frames. Although existing methods have excelled in producing coherent and expressive stories, they remain largely emotion-neutral, focusing on what subject appears in a story while overlooking how emotions shape narrative interpretation and visual presentation. As stories are intended to engage audiences emotionally, we introduce emotion-aware story generation, a new task that aims to generate subject-consistent visual stories with explicit emotional directions. This task is challenging due to the abstract nature of emotions, which must be grounded in concrete visual elements and consistently expressed across a narrative through visual composition. To address these challenges, we propose EmoStory, a two-stage framework that integrates agent-based story planning and region-aware story generation. The planning stage transforms target emotions into coherent story prompts with emotion agent and writer agent, while the generation stage preserves subject consistency and injects emotion-related elements through region-aware composition. We evaluate EmoStory on a newly constructed dataset covering 25 subjects and 600 emotional stories. Extensive quantitative and qualitative results, along with user studies, show that EmoStory outperforms state-of-the-art story generation methods in emotion accuracy, prompt alignment, and subject consistency.

Problem

• emotion-aware story generation
• subject consistency
• visual narrative
• emotional direction
• story generation

Innovation

• emotion-aware story generation
• agent-based planning
• region-aware composition
• subject consistency
• visual narrative