Guideline-Grounded Evidence Accumulation for High-Stakes Agent Verification

2026-03-03

AI Summary

This work addresses the critical challenge of verifying decisions made by large language model (LLM) agents in high-stakes scenarios, where existing approaches suffer from insufficient domain knowledge integration and poor calibration. To this end, the authors propose GLEAN, a novel verification framework that systematically incorporates expert guidelines into the validation process for the first time. GLEAN translates guidelines into correctness signals through trajectory-aware parsing, accumulates evidence from multiple guidelines along the agent’s decision path, and applies Bayesian logistic regression for calibrated confidence estimation. Additionally, it introduces an uncertainty-driven active verification mechanism that dynamically expands guideline coverage and performs discrepancy checks. Evaluated on the MIMIC-IV clinical diagnosis task, GLEAN substantially outperforms baseline methods, achieving a 12% improvement in AUROC and a 50% reduction in Brier score, with validation from clinical experts.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Multiagent Systems: Adversarial Agents
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Agentic search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

As LLM-powered agents have been used for high-stakes decision-making, such as clinical diagnosis, it becomes critical to develop reliable verification of their decisions to facilitate trustworthy deployment. Yet, existing verifiers usually underperform owing to a lack of domain knowledge and limited calibration. To address this, we establish GLEAN, an agent verification framework with Guideline-grounded Evidence Accumulation that compiles expert-curated protocols into trajectory-informed, well-calibrated correctness signals. GLEAN evaluates the step-wise alignment with domain guidelines and aggregates multi-guideline ratings into surrogate features, which are accumulated along the trajectory and calibrated into correctness probabilities using Bayesian logistic regression. Moreover, the estimated uncertainty triggers active verification, which selectively collects additional evidence for uncertain cases via expanding guideline coverage and performing differential checks. We empirically validate GLEAN with agentic clinical diagnosis across three diseases from the MIMIC-IV dataset, surpassing the best baseline by 12% in AUROC and 50% in Brier score reduction, which confirms the effectiveness in both discrimination and calibration. In addition, the expert study with clinicians recognizes GLEAN's utility in practice.

Problem

• agent verification
• high-stakes decision-making
• clinical diagnosis
• reliable verification
• domain knowledge

Innovation

• Guideline-Grounded Verification
• Evidence Accumulation
• Bayesian Calibration
• Active Verification
• LLM Agent Validation