What Must Generalist Agents Remember?

2026-06-17

AI Summary

This work addresses the challenge faced by general-purpose agents operating in multi-task environments where observation bottlenecks restrict perceptual information and mutually exclusive optimal actions must be selected. Under such conditions, relying solely on current observations precludes near-optimal behavior. The paper introduces the "Memory Separation Theorem," which formally characterizes the minimal memory requirements for cross-task generalization: agents must maintain distinguishable memory distributions at observation bottlenecks. Leveraging tools from information theory and reinforcement learning, the authors model observation bottlenecks and analyze memory distributions to prove that near-optimal policies necessarily depend on memory. Furthermore, they demonstrate that this memory enables approximate reconstruction of local environment dynamics, thereby facilitating domain identification and planning.

Abstract

This paper develops a formal account of what generalist agents must store in memory in order to act near-optimally across multiple environments and goals. It shows that when two domains share an observational bottleneck but require incompatible optimal actions, any uniformly near-optimal policy must induce distinct memory distributions at that bottleneck. The result yields a separation theorem: sufficiently successful agents cannot rely only on current state observations, but must preserve domain-relevant information in memory. The paper further shows that if an agent's memory contains enough information to estimate values for related goals, then that memory can be used to approximately reconstruct the agent's local transition dynamics. Together, these results characterize memory as the substrate that supports domain disambiguation, transition-model reconstruction, and planning for generalist agents.

Problem

• generalist agents
• memory
• near-optimality
• domain disambiguation
• transition dynamics

Innovation

• generalist agents
• memory representation
• domain disambiguation
• transition dynamics reconstruction
• near-optimal policy