SceneLoom: Communicating Data with Scene Context

2025-07-22

AI Summary

In data journalism and data-driven video, visualizations are often decoupled from real-world imagery, undermining narrative coherence and user engagement. To address this, we propose a narrative-intent-driven visualization–scene integration framework that establishes a tri-dimensional design mapping mechanism—coordinating visual, semantic, and data modalities. Leveraging joint vision-language models, our method extracts aligned features from both scene images and charts; it then achieves fine-grained fusion and dynamic transitions through spatial alignment, shape matching, layout consistency, and semantic binding. The system generates diverse design solutions that jointly preserve contextual expressiveness and data fidelity. User studies and evaluation on a curated case repository demonstrate significant improvements in authoring efficiency and narrative effectiveness. Our approach provides a scalable, technically grounded pathway toward immersive, data-driven storytelling.

Technology Category

• Computer Vision: Multi-modal Vision
• Data Mining & Knowledge Management: Mining of Visual, Multimedia & Multimodal Data
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Web Mining and Content Analysis: Web data visualization
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Security and Privacy: Data transparency and provenance

Abstract

In data-driven storytelling contexts such as data journalism and data videos, data visualizations are often presented alongside real-world imagery to support narrative context. However, these visualizations and contextual images typically remain separated, limiting their combined narrative expressiveness and engagement. Achieving this is challenging due to the need for fine-grained alignment and creative ideation. To address this, we present SceneLoom, a Vision-Language Model (VLM)-powered system that facilitates the coordination of data visualization with real-world imagery based on narrative intents. Through a formative study, we investigated the design space of coordination relationships between data visualization and real-world scenes from the perspectives of visual alignment and semantic coherence. Guided by the derived design considerations, SceneLoom leverages VLMs to extract visual and semantic features from scene images and data visualization, and perform design mapping through a reasoning process that incorporates spatial organization, shape similarity, layout consistency, and semantic binding. The system generates a set of contextually expressive, image-driven design alternatives that achieve coherent alignments across visual, semantic, and data dimensions. Users can explore these alternatives, select preferred mappings, and further refine the design through interactive adjustments and animated transitions to support expressive data communication. A user study and an example gallery validate SceneLoom's effectiveness in inspiring creative design and facilitating design externalization.

Problem

• Bridging data visualization and real-world imagery for narrative enhancement
• Achieving fine-grained alignment between visuals and semantic context
• Automating creative design mapping for expressive data communication

Innovation

• VLM-powered system aligns data with imagery
• Extracts visual and semantic features for mapping
• Generates expressive design alternatives for coherence