DEEPCHART: How Far are LLMs from Faithful Data-Science Chart Generation?

2026-08-27

AI Summary

研究通过DEEPCHART评估了大语言模型在生成忠实数据科学图表时面临的问题，采用提取-推理-可视化流程检测数据层面的错误。

Abstract

Faithful chart generation in real-world data-science workflows requires grounding visualizations in scattered evidence, computing chart-ready quantities, and rendering them accurately. Modern LLMs can produce visually plausible, instruction-compliant charts, yet data-level hallucinations remain difficult to detect in long, noisy, and multimodal contexts. To measure this gap, we introduce DEEPCHART, an expert-annotated benchmark of 1,482 task-conditioned chart-generation instances drawn from real-world scientific papers, financial filings, and ecosystem reports. DEEPCHART formulates chart generation as an Extract--Reason--Visualize pipeline and evaluates source-data extraction, derived-data reasoning, and chart rendering stage by stage. Experiments with state-of-the-art models show that visually plausible charts often conceal data-level hallucinations, with extraction and reasoning errors common in realistic long and multimodal settings. These findings suggest that larger context windows alone are insufficient; faithful chart generation also requires reliable evidence extraction and quantitative reasoning before rendering. Our benchmark and associated resources are available at https://github.com/tangdouer1005/DeepChart.

Problem

• chart generation
• data-level hallucinations
• multimodal contexts

Innovation

• DEEPCHART
• Extract-Reason-Visualize pipeline
• data-level hallucinations
• evidence extraction
• quantitative reasoning