ChartWalker: Benchmarking the Cross-Chart RAG Task

2026-06-22

AI Summary

Existing cross-chart retrieval-augmented generation (RAG) benchmarks are limited by high lexical overlap between queries and evidence and inconsistent reasoning chains, hindering their ability to support complex multimodal analysis. This work proposes ChartWalker, a framework that constructs chart-oriented hierarchical knowledge graphs and employs a structure-aware sampling algorithm to explicitly control query difficulty and granularity, thereby synthesizing high-quality question-answer pairs with multi-hop reasoning paths. Leveraging this approach, we introduce ChartWalker-Bench, the first comprehensive cross-chart RAG benchmark, which exposes significant performance bottlenecks in current state-of-the-art methods. To facilitate future research, we also release ChartWalker-Agent, an open-source agent baseline designed for this challenging task.

Abstract

Cross-Chart Retrieval-Augmented Generation (RAG) is critical for complex multi-modal analytical tasks in scientific, business, and political domains. However, existing benchmarks either focus on tables, which are well-structured and textualized, or generate cross-chart questions by simply extracting key points, which often induces lexical overlap between queries and evidence and yields logically inconsistent reasoning chains. To address this, we introduce ChartWalker, a novel framework for constructing challenging cross-chart RAG tasks. ChartWalker features a hierarchical knowledge graph construction method tailored to charts, which organizes entities and relations by granularity to preserve analytical structure. We then propose a structure-aware sampling algorithm that synthesizes semantically coherent, multi-hop reasoning paths, enabling explicit control over query difficulty and granularity for QA generation. Built with this framework, we release ChartWalker-Bench, a comprehensive benchmark spanning diverse domains and cross-chart query types. Extensive evaluations across major RAG paradigms reveal significant performance gaps, underscoring the benchmark's difficulty and utility. Furthermore, we provide ChartWalker-Agent, an agentic baseline to facilitate analysis and inspire future system design.

Problem

• Cross-Chart RAG
• benchmark
• multi-modal reasoning
• retrieval-augmented generation
• chart understanding

Innovation

• Cross-Chart RAG
• Hierarchical Knowledge Graph
• Structure-Aware Sampling
• Multi-Hop Reasoning
• Chart Understanding