Interactive Memory Learning for Long-Term Conversations

2026-09-15

AI Summary

为解决长期对话中静态记忆管理问题，提出ICML框架，通过互动式学习和强化学习机制动态调整记忆策略以优化响应质量。

Abstract

Recent advancements in large language models have significantly enhanced the capabilities of agents in modeling long-term conversations. Despite these successes, existing approaches typically adopt a static heuristic paradigm, where information is passively archived without adaptive memory valuation. Consequently, these methods fail to self-evolve or align their memory management with evolving user needs. To address this, we propose ICML (InteraCtive Memory Learning), a multi-agent framework that transforms the memory mechanism from a passive archive into a learnable, interactive memory policy. Specifically, we first employ a session synthesis pipeline to generate expert data, facilitating rapid test-time adaptation in unseen scenarios. Building on this, ICML utilizes an online reinforcement learning mechanism where a Planner agent selectively encodes high-value information and a Trigger agent dynamically retrieves it to optimize response quality, whereby the two agents co-evolve through continuous interaction feedback. Crucially, both agents are synchronized through a delayed reward mechanism that propagates future feedback back to earlier storage decisions, ensuring memory policies are precisely aligned with user expectations. Experimental results demonstrate that ICML significantly outperforms strong baselines, exhibiting the unique capability to continuously improve response quality as interactions accumulate.

Problem

• long-term conversations
• static heuristic paradigm
• adaptive memory valuation
• memory management
• user needs

Innovation

• Interactive Memory Learning
• Multi-agent Framework
• Reinforcement Learning
• Delayed Reward Mechanism