RÉCITKIT: A Spatial Toolkit for Designing and Evaluating Human-Centered Immersive Data Narratives

2025-08-26

AI Summary

Spatial computing presents new opportunities for immersive data storytelling, yet lacks systematic design principles and development support. This paper introduces the first immersive data storytelling framework tailored for head-mounted displays (e.g., Apple Vision Pro), extending conventional narrative visualization into 3D space. The framework supports interactive dashboard construction, dynamic binding of data attributes to 3D scenes, synchronized voice/text narration, and physics-based interactions—including gaze, pinch, and navigation gestures. It integrates spatiotemporal data filtering, hierarchical drill-down, and multimodal narrative mechanisms. Evaluated on the Napoleonic 1812 campaign case study, the framework was assessed by 21 users. Results demonstrate significant improvements in developer productivity and substantial enhancements in users’ depth of understanding and engagement with complex spatiotemporal data.

Technology Category

• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Data Mining & Knowledge Management: Mining of Spatial, Temporal or Spatio-Temporal Data
• Humans and AI: Game Design — Procedural Content Generation & Storytelling

Application Category

• Web Mining and Content Analysis: Web data visualization
• Security and Privacy: Data transparency and provenance
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Spatial computing presents new opportunities for immersive data storytelling, yet there is limited guidance on how to build such experiences or adapt traditional narrative visualizations to this medium. We introduce a toolkit, RÉCITKIT for supporting spatial data narratives in head-mounted display (HMD) environments. The toolkit allows developers to create interactive dashboards, tag data attributes as spatial assets to 3D models and immersive scenes, generate text and audio narratives, enabling dynamic filtering, and hierarchical drill-down data discoverability. To demonstrate the utility of the toolkit, we developed Charles Minard's historical flow map of Napoleon's 1812 campaign in Russia as an immersive experience on Apple Vision Pro. We conducted a preliminary evaluation with 21 participants that comprised two groups: developers, who evaluated the toolkit by authoring spatial stories and consumers, who provided feedback on the Minard app's narrative clarity, interaction design, and engagement. Feedback highlighted how spatial interactions and guided narration enhanced insight formation, with participants emphasizing the benefits of physical manipulation (e.g., gaze, pinch, navigation) for understanding temporal and geographic data. Participants also identified opportunities for future enhancement, including improved interaction affordance visibility, customizable storytelling logic, and integration of contextual assets to support user orientation. These findings contribute to the broader discourse on toolkit-driven approaches to immersive data storytelling across domains such as education, decision support, and exploratory analytics.

Problem

• Developing tools for immersive spatial data storytelling
• Adapting traditional visualizations to head-mounted display environments
• Enhancing data insight through spatial interactions and narration

Innovation

• Toolkit for spatial data narratives in HMD environments
• Enables interactive dashboards and dynamic data filtering
• Supports physical manipulation like gaze and pinch interactions