LEDGER: Claim-to-Evidence Trace Graphs for Auditing LLM Agents

2026-08-18

AI Summary

为解决大语言模型代理输出的审核问题，提出LEDGER系统，通过构建分层追踪图来连接声明与支持性操作、工件和检查，以提高审计效率。

Abstract

Large language model (LLM) agents can now carry out long-horizon technical workflows involving complex tool use, code execution, file edits, and generated artifacts. As agents do more work faster, the productivity bottleneck shifts from producing outputs to auditing whether those outputs are correct and trustworthy. Agent observability systems make fine-grained execution events visible, but visibility alone still leaves reviewers to reconstruct which actions, artifacts, and validation steps matter for a particular conclusion. We introduce LEDGER - Layered Evidence and Decision Graphs for Execution Review, a tracing and review system that builds layered trace graphs over observed agent sessions. LEDGER preserves Trace Records while grouping them into Evidence Nodes and Workflow Nodes, representing artifacts as evidence anchors, and adding typed semantic edges that connect claims to supporting actions, artifacts, and checks. Through data-analysis and coding examples, we show how the resulting traces expose workflow decisions, artifact lineage, repair steps, validation coverage, and claim-support paths for evidence-centered audit.

Problem

• Large Language Model (LLM) Agents
• Auditing
• Workflow

Innovation

• LEDGER
• Layered Trace Graphs
• Evidence Nodes
• Workflow Nodes
• Semantic Edges