Remember the Decision, Not the Description: A Rate-Distortion Framework for Agent Memory

2026-05-11

AI Summary

This work addresses the challenge that long-horizon language agents struggle to retain decision-critical historical distinctions under limited memory, often accumulating redundant descriptive memories. The authors formulate memory modeling as a decision-centric rate-distortion problem and introduce the core principle of “remembering decisions, not descriptions”: memory value is quantified by decision-quality loss, enabling dynamic state-space partitioning and the derivation of decision-oriented memory forgetting boundaries and a memory–distortion frontier. Building on this framework, they propose DeMem, an online learning algorithm that refines state partitions only upon encountering decision conflicts, thereby guaranteeing a near-minimax regret bound. Experiments on synthetic diagnostic and long-horizon dialogue tasks demonstrate that DeMem significantly improves decision performance under identical memory budgets, validating the effectiveness and novelty of the proposed approach.

Abstract

Long-horizon language agents must operate under limited runtime memory, yet existing memory mechanisms often organize experience around descriptive criteria such as relevance, salience, or summary quality. For an agent, however, memory is valuable not because it faithfully describes the past, but because it preserves the distinctions between histories that must remain separated under a fixed budget to support good decisions. We cast this as a decision-centric rate-distortion problem, measuring memory quality by the loss in achievable decision quality induced by compression. This yields an exact forgetting boundary for what can be safely forgotten, and a memory-distortion frontier characterizing the optimal tradeoff between memory budget and decision quality. Motivated by this decision-centric view of memory, we propose DeMem, an online memory learner that refines its partition only when data certify that a shared state would induce decision conflict, and prove near-minimax regret guarantees. On both controlled synthetic diagnostics and long-horizon conversational benchmarks, DeMem yields consistent gains under the same runtime budget, supporting the principle that memory should preserve the distinctions that matter for decisions, not descriptions.

Problem

• agent memory
• rate-distortion
• decision quality
• memory compression
• long-horizon agents

Innovation

• decision-centric memory
• rate-distortion theory
• DeMem
• memory compression
• minimax regret