Visual Writing: Writing by Manipulating Visual Representations of Stories

2024-10-09

arXiv.org

AI Summary

To address challenges in narrative writing—including difficulty tracking narrative elements, ambiguity in expressing editing intent, and inefficiency in exploring story variants—this paper introduces “visual writing,” a novel paradigm that models time, entities, events, and locations as multidimensional visual representations. It enables authors to directly manipulate narrative structure via a graphical interface, rather than editing text alone. We formally define the concept of visual writing, propose a taxonomy of visualization-based narrative representations tailored for editing tasks, and design an interactive system supporting bidirectional coupling between textual content and visual encodings of entities, actions, spatial layout, and temporal sequencing. An empirical study with eight professional writers demonstrates significant improvements in paragraph localization, cross-element relationship tracking, articulation of editing intent, and generation of narrative variants—while also stimulating creative ideation. The work advances the integration of visualization into the end-to-end creative writing workflow.

Technology Category

• Cognitive Modeling & Cognitive Systems: Computational Creativity
• Computer Vision: Visual Reasoning & Symbolic Representations
• Humans and AI: Game Design — Procedural Content Generation & Storytelling

Application Category

• Web Mining and Content Analysis: Web data visualization
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

We introduce"visual writing", an approach to writing stories by manipulating visuals instead of words. Visual writing relies on editable visual representations of time, entities, events, and locations to offer representations more suited to specific editing tasks. We propose a taxonomy for these representations and implement a prototype software supporting the visual writing workflow. The system allows writers to edit the story by alternating between modifying the text and manipulating visual representations to edit entities, actions, locations, and order of events. We evaluate this workflow with eight creative writers and find visual writing can help find specific passages, keep track of story elements, specify edits, and explore story variations in a way that encourages creativity.

Problem

• Supporting narrative writing with visual story elements
• Automating text edits via interactive visualizations
• Enhancing creativity through visual story planning

Innovation

• Text editor visualizes entity interaction graphs
• Interactive visuals suggest narrative text edits
• Visuals aid planning and tracking story elements