Doc2Chart: Intent-Driven Zero-Shot Chart Generation from Documents

2025-07-20

AI Summary

This paper introduces the novel task of *intent-driven document-to-chart generation*, which aims to automatically produce accurate, intent-consistent visualizations from long documents and natural language intents—without manual content selection and under zero-shot settings. Methodologically, we propose an unsupervised two-stage framework: (1) intent decomposition and key data extraction using large language models; and (2) heuristic-guided chart type selection followed by executable visualization code generation. To address limitations of conventional vision-based metrics, we design a novel attribution-based chart-data fidelity metric for quantitative evaluation. Extensive experiments on 1,242 samples from finance and scientific domains demonstrate that our approach achieves 9.0% and 17.0% absolute improvements over the strongest baseline in chart-data accuracy and chart-type selection accuracy, respectively.

Technology Category

• Data Mining & Knowledge Management: Data Visualization & Summarization
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Computer Vision: Visual Reasoning & Symbolic Representations

Application Category

• Web Mining and Content Analysis: Web data visualization
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Large Language Models (LLMs) have demonstrated strong capabilities in transforming text descriptions or tables to data visualizations via instruction-tuning methods. However, it is not straightforward to apply these methods directly for a more real-world use case of visualizing data from long documents based on user-given intents, as opposed to the user pre-selecting the relevant content manually. We introduce the task of intent-based chart generation from documents: given a user-specified intent and document(s), the goal is to generate a chart adhering to the intent and grounded on the document(s) in a zero-shot setting. We propose an unsupervised, two-staged framework in which an LLM first extracts relevant information from the document(s) by decomposing the intent and iteratively validates and refines this data. Next, a heuristic-guided module selects an appropriate chart type before final code generation. To assess the data accuracy of the generated charts, we propose an attribution-based metric that uses a structured textual representation of charts, instead of relying on visual decoding metrics that often fail to capture the chart data effectively. To validate our approach, we curate a dataset comprising of 1,242 $<$intent, document, charts$>$ tuples from two domains, finance and scientific, in contrast to the existing datasets that are largely limited to parallel text descriptions/ tables and their corresponding charts. We compare our approach with baselines using single-shot chart generation using LLMs and query-based retrieval methods; our method outperforms by upto $9$ points and $17$ points in terms of chart data accuracy and chart type respectively over the best baselines.

Problem

• Generating charts from documents using user intents
• Extracting and validating data without manual selection
• Improving accuracy in zero-shot chart generation

Innovation

• LLM extracts data by decomposing user intent
• Heuristic-guided module selects chart type
• Attribution-based metric assesses data accuracy