DataWink: Reusing and Adapting SVG-based Visualization Examples with Large Multimodal Models

2025-07-23

AI Summary

To address the challenge non-expert users face in creating aesthetically pleasing data visualizations, this paper proposes an interactive SVG visualization generation method powered by large multimodal models (LMMs). The approach parses exemplar SVGs to extract structured data-to-visual mappings and stylistic specifications, constructing an editable intermediate representation. It integrates a conversational agent with on-demand generated visualization controls, enabling users to express customization intents in natural language and adjust designs interactively in real time. Key contributions include: (1) the first LMM-driven semantic parsing and programmable reuse of SVGs, jointly optimizing aesthetic fidelity and interactive flexibility; and (2) empirical validation via user studies demonstrating significant improvements in replication accuracy and exploratory efficiency—outperforming baselines across learnability, personalized control, and output quality.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Visual Reasoning & Symbolic Representations
• Data Mining & Knowledge Management: Data Visualization & Summarization

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Web Mining and Content Analysis: Web data visualization

Abstract

Creating aesthetically pleasing data visualizations remains challenging for users without design expertise or familiarity with visualization tools. To address this gap, we present DataWink, a system that enables users to create custom visualizations by adapting high-quality examples. Our approach combines large multimodal models (LMMs) to extract data encoding from existing SVG-based visualization examples, featuring an intermediate representation of visualizations that bridges primitive SVG and visualization programs. Users may express adaptation goals to a conversational agent and control the visual appearance through widgets generated on demand. With an interactive interface, users can modify both data mappings and visual design elements while maintaining the original visualization's aesthetic quality. To evaluate DataWink, we conduct a user study (N=12) with replication and free-form exploration tasks. As a result, DataWink is recognized for its learnability and effectiveness in personalized authoring tasks. Our results demonstrate the potential of example-driven approaches for democratizing visualization creation.

Problem

• Enables non-experts to create custom visualizations easily
• Extracts data encoding from SVG examples using LMMs
• Maintains aesthetic quality while allowing design modifications

Innovation

• Uses large multimodal models for SVG extraction
• Combines conversational agent with on-demand widgets
• Interactive interface maintains aesthetic quality