Beyond Single-Turn Confidence: Trajectory-Adapted Uncertainty Quantification for LLM Agents

2026-08-11

AI Summary

This study addresses the challenge of quantifying uncertainty in multi-turn interactive agents, where existing language model uncertainty quantification (UQ) methods—primarily designed for single-turn outputs—struggle to support trajectory-level decision-making. The work presents the first systematic evaluation of three classes of single-turn UQ approaches adapted to multi-turn tool-use tasks at the trajectory level: white-box scoring based on action token probabilities, black-box self-consistency scoring via resampled trajectories (including trajectory equivalence and action-set consistency), and reflexive scoring based on model self-assessment. Experiments on BFCL-v4 and τ²-bench show that black-box self-consistency methods perform best overall, with trajectory equivalence and action-set consistency yielding superior results; reflexive scoring offers an efficient baseline; and token-probability scoring proves highly sensitive to aggregation strategies. The findings highlight the critical influence of aggregation schemes, consistency definitions, and computational cost on UQ effectiveness.

Abstract

Uncertainty quantification (UQ) methods for language models are typically evaluated on single-turn outputs, where uncertainty is attached to one generated answer. For LLM agents, however, the unit of observation is an interactive trajectory, where the model can ask clarifying questions, call tools, update state, and make intermediate decisions whose errors propagate to the final outcome. We study whether three common families of single-turn UQ methods transfer to this setting. Across five LLMs and four multi-turn tool-use datasets from BFCL-v4 and $τ^2$-bench, we evaluate white-box scorers based on action-token probabilities, black-box consistency scorers based on resampled trajectories, and reflexive scorers based on model self-assessment of the trajectory. We find that transfer is often useful but uneven. Token-probability scores are highly sensitive to the choice of aggregator used across turns, reflexive scores provide the strongest low-cost baseline in most evaluated settings, and black-box self-consistency is often the strongest UQ family, with trajectory-equivalence and action-set consistency typically ranking highest among its variants. These results suggest that UQ methods developed for single generations should be revalidated at the trajectory level, with careful attention to the consistency measurement, aggregator choice, and computational budget.

Problem

• Uncertainty Quantification
• LLM Agents
• Interactive Trajectory
• Multi-turn Tool-use
• Error Propagation

Innovation

• Uncertainty Quantification
• LLM Agents
• Trajectory-Level Evaluation
• Self-Consistency
• Tool-Use Tasks