ECHO: Learning Epistemically Adaptive Language Agents with Turn-Level Credit

2026-06-28

AI Summary

Existing multi-turn language agents lack explicit modeling of cognitive states, hindering their ability to dynamically assess uncertainty and make adaptive decisions. This work proposes the Epistemic Decision Process (EDP) framework, which formulates information seeking as a sequential decision-making problem under belief states, and introduces the ECHO algorithm featuring a posterior-sensitive per-turn reward mechanism to enable epistemic credit assignment. The study formally defines the problem of cognitively adaptive decision-making for the first time, exposes the limitations of trajectory-level rewards, and advances a belief-driven approach to behavioral optimization. Evaluated on the Clue Selector Game benchmark, the proposed method substantially improves information acquisition efficiency, resolution, and calibration while outperforming baselines such as GRPO, all with negligible generation of explicit reasoning traces.

Abstract

What does it mean for a language agent to be adaptive? Effective multi-turn agents must decide what information to seek, how to use new evidence, and when they are certain enough to act. We introduce Epistemic Decision Processes (EDPs), a belief-state formulation of multi-turn information seeking in which actions produce external observations that update the agent's posterior over a latent task variable. EDPs make epistemic adaptivity explicit: good policies choose actions that are useful under the current belief, not merely those that correlate with eventual success. We prove that belief-agnostic policies can suffer errors that compound exponentially over the horizon, and that aggregate trajectory returns can fail to identify the per-turn Bayesian advantage needed for epistemic credit. We then introduce ECHO (Epistemic Credit for History-Conditioned Optimization), a practical clipped policy-gradient objective that assigns turn-level credit using posterior-sensitive rewards. In the Clue Selector Game, a novel controlled evidence-seeking benchmark, we show that ECHO substantially improves resolution, information gain, and efficiency over trajectory-level GRPO, and matches or exceeds frontier baselines on epistemic metrics such as grounding, recovery, and calibration while producing almost no visible reasoning text.

Problem

• epistemic adaptivity
• multi-turn information seeking
• belief-state
• language agents
• epistemic credit

Innovation

• Epistemic Adaptivity
• Epistemic Decision Processes
• Turn-Level Credit Assignment
• Posterior-Sensitive Rewards
• ECHO