Problem
Research questions and friction points this paper is trying to address.
agent evaluation
state transition
step-level diagnostics
belief update
LLM evaluator
Agent Step Value: State-Transition Measurement with State-Grounded LLM Evaluators
🤖 AI Summary
This work addresses the limitation of existing agent evaluation methods, which predominantly rely on final answers or trajectory-level scores and thus struggle to pinpoint the critical actions that drive state transitions toward favorable outcomes. The authors propose the Agent Step Value (ASV) framework, which employs a state-anchored LLM evaluator to quantitatively score the state transition induced by each individual action, enabling the first unsupervised, step-level belief diagnosis. By decoupling reasoning rationales from option scoring and integrating log-probabilities from stateless LLMs with metrics such as Bayesian surprise, entropy change, and gold-margin gain—augmented by redacted state projection and rationale-conditioned protocols—the method identifies constructive and destructive belief turning points across 1,100 steps and 2,200 states in 100 open-ended QA tasks, achieving an average gold-margin gain of −2.335 and Bayesian surprise of 2.693.
📝 Abstract
Most agent evaluations collapse a multi-step trace into a final answer, a success flag, or a trajectory-level score. These aggregates obscure the diagnostic question developers need most: which action changed the state in a useful direction? We introduce Agent Step Value (ASV), a state-transition measurement framework that scores each observed action by the change it induces in a state-grounded evaluator's distribution over fixed candidate outcomes. ASV renders redacted before/after state projections, uses a stateless LLM evaluator to assign candidate log scores, and reports both gold-free belief diagnostics and offline oracle validation metrics. A label-free rationale pass separates evaluator deliberation from one-token option scoring, preserving candidate likelihoods while exposing leakage and floor-score events. On 100 reviewed open-QA evidence-seeking tasks with live PubMed retrieval, a partially live DeepSeek actor, and DeepSeek log-probability scoring, ASV evaluates 1,100 steps and 2,200 states. Under the fixed-layout rationale-conditioned protocol, mean gold-margin gain is -2.335 (trajectory-bootstrap 95\% CI [-3.395, -1.272]), entropy movement is 0.000, and mean Bayesian surprise is 2.693. ASV therefore localizes constructive and destructive belief pivots that final-answer scores and entropy-only step metrics miss. We release the standalone ASV Eval toolkit.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Agent Step Value
state-transition measurement
state-grounded LLM evaluator
rationale-conditioned scoring
belief diagnostics
🔎 Similar Papers
No similar papers found.
💼 Related Jobs
