🤖 AI Summary
This work addresses the challenges large language models face in generating accurate and readable statistical visualizations, stemming from existing datasets' lack of full alignment among code, data context, and question-answer pairs. The authors propose a structured, multi-stage workflow that decomposes chart generation into verifiable steps—data filtering, plotting proposal, code synthesis, rendering, and validation-driven refinement—and introduces a rendering feedback mechanism to transform the task from one-shot code generation into an iterative, verifiable process. The approach jointly produces charts, code, contextual metadata, natural language descriptions, and associated question-answer pairs, yielding a benchmark of 1,500 visualizations (spanning 24 chart types) and 30,003 QA pairs across 74 UCI datasets. Evaluation of 16 vision-language models demonstrates that this framework effectively exposes their limitations in numerical extraction, comparison, and reasoning, enabling fine-grained assessment of visual reasoning capabilities.
📝 Abstract
Generating diverse, readable statistical charts from tabular data remains challenging for LLMs, as many failures become apparent after rendering and are not detectable from data or code alone. Existing chart datasets also rarely provide fully aligned artifacts, such as executable code, dataset context, and question-answer pairs. We present a structured LLM-based workflow that decomposes chart generation into dataset screening, plot proposal, code synthesis, rendering, validation-driven refinement, description generation, and question-answer generation. By incorporating rendered-output validation, the workflow addresses visualization-specific failure modes such as readability and semantic mismatch. It treats chart generation as an inspectable process rather than a one-shot prompt-to-code task, retaining each chart with its code, dataset context, description, and question-answer pairs. Applied to UCI datasets, the workflow produces 1,500 charts from 74 datasets, spanning 24 chart families and paired with 30,003 question-answer pairs. We evaluate 16 multimodal LLMs (MLLMs) on these chart-question pairs. The results show that chart-syntax questions are nearly saturated, while value extraction, comparison, and reasoning remain more challenging, illustrating the workflow's utility for diagnostic studies of chart-grounded multimodal reasoning.