Graph of Verification: Structured Verification of LLM Reasoning with Directed Acyclic Graphs

2025-06-14

AI Summary

To address the insufficient faithfulness and precision in multi-step reasoning verification for large language models (LLMs), this paper proposes the first directed acyclic graph (DAG)-based structured verification framework. It explicitly models the reasoning process as a topologically ordered node dependency graph, enabling modular verification at customizable granularities—either proposition-level or paragraph-level. We design a topological-order-guided stepwise verification mechanism and a node-block context injection strategy to jointly ensure faithfulness to the overall reasoning path and precision in local judgments. Evaluated on the Number Triangle Summation and ProcessBench benchmarks, our method achieves significant improvements in verification accuracy, faithfulness, and fine-grained error localization, consistently outperforming end-to-end baseline approaches across all metrics.

Technology Category

• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Verifying the reliability of complex, multi-step reasoning in Large Language Models (LLMs) remains a fundamental challenge, as existing methods often lack both faithfulness and precision. To address this issue, we propose the Graph of Verification (GoV) framework. GoV offers three key contributions: First, it explicitly models the underlying deductive process as a directed acyclic graph (DAG), whether this structure is implicit or explicitly constructed. Second, it enforces a topological order over the DAG to guide stepwise verification. Third, GoV introduces the notion of customizable node blocks, which flexibly define the verification granularity, from atomic propositions to full paragraphs, while ensuring that all requisite premises derived from the graph are provided as contextual input for each verification unit. We evaluate GoV on the Number Triangle Summation task and the ProcessBench benchmark with varying levels of reasoning complexity. Experimental results show that GoV substantially improves verification accuracy, faithfulness, and error localization when compared to conventional end-to-end verification approaches. Our code and data are available at https://github.com/Frevor/Graph-of-Verification.

Problem

• Verifying multi-step LLM reasoning lacks faithfulness and precision
• Modeling deductive processes as directed acyclic graphs for structured verification
• Improving verification accuracy and error localization in complex reasoning tasks

Innovation

• Models reasoning as directed acyclic graph
• Enforces topological order for verification
• Uses customizable node blocks for granularity