VeriGraph: Towards Verifiable Data-Analytic Agents

2026-06-15

AI Summary

This work addresses the opacity of reasoning in large language model (LLM) agents performing data-intensive analysis, which hinders the verifiability of their conclusions. To resolve this, the authors propose VeriGraph—a traceable neuro-symbolic reasoning framework that constructs an explicit heterogeneous evidence directed acyclic graph (DAG) to unify raw data, variables, computational results, and natural language claims. VeriGraph introduces three evidence expansion primitives—computation, anchoring, and derivation—to enable structural traceability via graph reachability and incorporates claim-level evidence evaluation to quantify semantic support. Experimental results demonstrate that VeriGraph achieves state-of-the-art performance across four benchmarks, with its 8B variant attaining a claim-level anchoring accuracy of 87.61%, substantially enhancing the auditability and reproducibility of model outputs.

Abstract

LLM-based agents have demonstrated strong capabilities in data-intensive analytical tasks, yet their outputs are rarely verifiable: a reliance on linear text trajectories makes their reasoning difficult to audit. In particular, deterministic computations over raw data and semantic deductions over natural-language claims are often entangled in an unstructured stream, leaving numerical conclusions hard to reproduce and qualitative judgments hard to inspect. To address this, we propose VeriGraph, a traceable neuro-symbolic reasoning framework that enables agents to construct an explicit heterogeneous evidence directed acyclic graph (DAG) during execution. VeriGraph introduces three evidence-expansion primitives, namely computational, grounding, and derivational expansion, to connect raw data, interpreter variables, computed results, and natural-language claims in a unified graph. Under this formulation, structural traceability is reduced to graph reachability from raw data sources to terminal claims, while semantic support is measured by claim-level evidence evaluation. To improve graph construction, we further design a graph-based policy optimization strategy with a composite reward that jointly supervises answer correctness, computational integrity, and derivational coherence. Experiments on four benchmarks show that VeriGraph-8B achieves the highest overall score among all baselines. More importantly, VeriGraph produces auditable evidence graphs with substantially stronger claim grounding, achieving a 87.61\% Grounding Rate under our claim-level evidence support evaluation. These results suggest that explicit evidence-graph construction is a promising path toward verifiable data-analytic agents. Our code is available at https://github.com/ignorejjj/VeriGraph.

Problem

• verifiability
• data-analytic agents
• reasoning traceability
• evidence grounding
• LLM-based reasoning

Innovation

• verifiable reasoning
• evidence graph
• neuro-symbolic framework
• traceable LLM agents
• claim grounding