Leveraging Foundation Models for Crafting Narrative Visualization: A Survey

2024-01-25

AI Summary

Narrative visualization of complex data remains challenging in terms of interpretability and engagement. Method: We conduct a systematic literature review of 66 papers to propose the first end-to-end, four-stage reference model—Analysis, Narrative, Visualization, and Interaction—and distill eight core tasks, including insight extraction and author assistance. We further introduce a unified technical framework integrating large language models, multimodal understanding and generation, data insight mining, and human-AI collaboration, and systematically evaluate performance boundaries and challenges across tasks. Contribution/Results: We construct a structured knowledge graph that clarifies technological applicability scopes and open research questions, delivering an actionable roadmap and evaluation guidelines for researchers and practitioners in narrative visualization powered by foundation models.

Technology Category

• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models
• Data Mining & Knowledge Management: Data Visualization & Summarization
• Computer Vision: Language and Vision

Application Category

• Web Mining and Content Analysis: Web data visualization
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Narrative visualization transforms data into engaging stories, making complex information accessible to a broad audience. Foundation models, with their advanced capabilities such as natural language processing, content generation, and multimodal integration, hold substantial potential for enriching narrative visualization. Recently, a collection of techniques have been introduced for crafting narrative visualizations based on foundation models from different aspects. We build our survey upon 66 papers to study how foundation models can progressively engage in this process and then propose a reference model categorizing the reviewed literature into four essential phases: Analysis, Narration, Visualization, and Interaction. Furthermore, we identify eight specific tasks (e.g. Insight Extraction and Authoring) where foundation models are applied across these stages to facilitate the creation of visual narratives. Detailed descriptions, related literature, and reflections are presented for each task. To make it a more impactful and informative experience for diverse readers, we discuss key research problems and provide the strengths and weaknesses in each task to guide people in identifying and seizing opportunities while navigating challenges in this field.

Problem

• Enhancing narrative visualization using foundation models
• Categorizing literature into Analysis, Narration, Visualization, Interaction
• Identifying tasks for foundation models in visual narratives

Innovation

• Foundation models enhance narrative visualization.
• Survey categorizes techniques into four phases.
• Identifies eight tasks for visual narratives.