Retrieval Augmented Comic Image Generation

2025-06-14

AI Summary

This work addresses the challenges of inter-frame inconsistency in character identity and attire, as well as monotonous pose variation, in comic-style image sequence generation. We propose a retrieval-augmented, region-controlled diffusion model. Our method integrates image retrieval, text–image alignment, region-conditioned modeling, and diffusion model fine-tuning. Key contributions include: (1) a retrieval-based character matching module that aligns textual prompts with character appearance using reference images; and (2) a region-wise character feature injection mechanism enabling localized control over facial features, clothing, and other semantic parts. Evaluated on multi-frame comic generation, our approach significantly improves character consistency and pose diversity, achieving state-of-the-art narrative coherence and visual vividness.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Humans and AI: Game Design — Procedural Content Generation & Storytelling
• Natural Language Processing: Generation

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

We present RaCig, a novel system for generating comic-style image sequences with consistent characters and expressive gestures. RaCig addresses two key challenges: (1) maintaining character identity and costume consistency across frames, and (2) producing diverse and vivid character gestures. Our approach integrates a retrieval-based character assignment module, which aligns characters in textual prompts with reference images, and a regional character injection mechanism that embeds character features into specified image regions. Experimental results demonstrate that RaCig effectively generates engaging comic narratives with coherent characters and dynamic interactions. The source code will be publicly available to support further research in this area.

Problem

• Ensures consistent character identity across comic frames
• Generates diverse and vivid character gestures
• Aligns text prompts with reference images for accuracy

Innovation

• Retrieval-based character assignment module
• Regional character injection mechanism
• Generates consistent expressive comic sequences