CalVerT: Augmenting Agents with Calibrated Verifier Telemetry Improves Action and Learning in Knowledge-Intensive Tasks

2026-06-19

AI Summary

This work addresses the tendency of large language model agents to exhibit overconfidence or perform redundant retrieval in knowledge-intensive question answering due to inaccurate assessment of answer credibility and evidential sufficiency. The authors propose Calibrated Verifier Telemetry (CalVerT), a novel approach that jointly injects calibrated confidence scores and external fact-checking signals into the agent’s internal state, enabling improved retrieval and reasoning decisions without requiring additional training. By enhancing the agent’s state representation, CalVerT facilitates coordinated control between retrieval and inference and seamlessly integrates with reinforcement learning frameworks. Evaluated on four question-answering benchmarks, the method significantly improves F1 scores, effectively reduces unnecessary retrieval steps, and prevents unsupported responses, demonstrating strong applicability in both trained and untrained settings.

Abstract

LLM agents in knowledge intensive question answering take retrieval and reasoning actions with incomplete knowledge about whether their current answer is uncertain, unsupported, or already complete. This produces two failure modes: committing to confident but unsupported answers, which hurts accuracy, and over-retrieving when the evidence in hand already suffices, resulting in wasted compute. To give agents a more complete picture of the state space they are operating in, we introduce calibrated verifier telemetry (CalVerT), which augments the agent's state with additional telemetry: a calibrated self-confidence score and a grounding verifier score. We show that CalVerT can improve agents in both training-free and training-based settings. On four QA benchmarks, we find that CalVerT raises F1 by triggering retrieval in cases where agents over-rely on parametric knowledge, while cutting redundant retrieval in cases where agents have sufficient context to answer. We show that CalVerT can augment existing QA frameworks without training. Moreover, CalVerT also improves trained systems: by simply augmenting an agent's state with telemetry, we observe improvements after reinforcement learning, as compared to an agent with identical training but no CalVerT telemetry.

Problem

• LLM agents
• knowledge-intensive tasks
• retrieval
• reasoning
• answer uncertainty

Innovation

• Calibrated Verifier Telemetry
• LLM agents
• retrieval-augmented QA
• self-confidence calibration
• grounding verification