AutoMem: Automated Learning of Memory as a Cognitive Skill

2026-07-01

AI Summary

This work addresses the limited autonomous memory management capabilities of large language models in long-horizon tasks, where they struggle to effectively decide when to store, retrieve, and organize knowledge. The study introduces, for the first time, a metacognitive memory mechanism into large language models, proposing a fully automated framework that models memory operations as trainable cognitive skills. This framework employs a dual-loop optimization strategy: an outer loop reconstructs memory structures through trajectory-based retrospective analysis using a strong model, while an inner loop fine-tunes memory strategies using multi-turn agent behaviors with effective memory usage as supervision. Evaluated on Crafter, MiniHack, and NetHack—three challenging long-horizon environments—the approach achieves 2–4× performance gains through memory optimization alone, enabling a 32B open-source model to match the performance of state-of-the-art systems such as Claude Opus 4.5 and Gemini 3.1 Pro Thinking.

Abstract

Memory expertise is a learned skill: knowing what to encode, when to retrieve, and how to organize knowledge--a capacity known in cognitive science as metamemory. We bring this perspective to LLMs by treating memory management as a trainable skill. We promote file-system operations to first-class memory actions alongside task actions, letting the model itself decide how to manage its memory. This memory skill improves along two axes: the structure that supports it (prompts, file schemas, action vocabulary), and the proficiency of the model exercising it. Both axes resist manual optimization: episodes in long-horizon tasks run for thousands of steps, and a single memory mistake can hide long before it surfaces, making human review of full trajectories impractical. We introduce AutoMem, a framework that automates both axes. In the first loop, a strong LLM reviews complete agent trajectories and iteratively revises the memory structure that shapes how the agent interacts with its memory files. In the second loop, the agent's own good memory decisions are identified from many episodes and used as training signal to sharpen the model's memory proficiency directly. Across three procedurally generated long-horizon games (Crafter, MiniHack, and NetHack), optimizing memory alone--without modifying the model's task-action behavior--improved the base agent's performance ~2x-4x, bringing a 32B open-weight model competitive with frontier systems such as Claude Opus 4.5 and Gemini 3.1 Pro Thinking. Our results show that memory management is an independently learnable skill, and a high-leverage objective yielding large gains on long-horizon tasks.

Problem

• metamemory
• memory management
• long-horizon tasks
• cognitive skill
• LLMs

Innovation

• metamemory
• memory management
• long-horizon tasks
• automated learning
• trajectory optimization