Collaborative Multi-Agent Optimization for Personalized Memory System

2026-03-13

AI Summary

This work addresses the limitation of existing personalized memory systems, where independently optimized agents lack coordination, hindering global performance. To overcome this, we propose CoMAM, a collaborative reinforcement learning framework that models multi-agent execution as a sequential Markov decision process, jointly optimizing local task rewards and global question-answering accuracy. CoMAM introduces an innovative cross-agent collaboration mechanism that dynamically allocates credit by aligning local and global rewards, and quantifies individual contributions through rank consistency to effectively align agent behaviors with the system’s overarching objective. Experimental results demonstrate that CoMAM significantly outperforms state-of-the-art methods on personalized memory tasks, validating the efficacy of collaborative reinforcement learning for multi-agent joint optimization.

Technology Category

• Multiagent Systems: Multiagent Learning
• Search and Optimization: Learning to Search
• Cognitive Modeling & Cognitive Systems: Agent Architectures

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Memory systems are crucial to personalized LLMs by mitigating the context window limitation in capturing long-term user-LLM conversations. Typically, such systems leverage multiple agents to handle multi-granular memory construction and personalized memory retrieval tasks. To optimize the system, existing methods focus on specializing agents on their local tasks independently via prompt engineering or fine-tuning. However, they overlook cross-agent collaboration, where independent optimization on local agents hardly guarantees the global system performance. To address this issue, we propose a Collaborative Reinforcement Learning Framework for Multi-Agent Memory Systems (CoMAM), jointly optimizing local agents to facilitate collaboration. Specifically, we regularize agents' execution as a sequential Markov decision process (MDP) to embed inter-agent dependencies into the state transition, yielding both local task rewards (e.g., information coverage for memory construction) and global rewards (i.e., query-answer accuracy). Then, we quantify each agent's contribution via group-level ranking consistency between local and global rewards, treating them as adaptive weights to assign global credit and integrate local-global rewards. Each agent is optimized by these integrated rewards, aligning local improvements with the global performance. Experiments show CoMAM outperforms leading memory systems, validating the efficacy of our proposed collaborative reinforcement learning for joint optimization.

Problem

• multi-agent optimization
• personalized memory system
• collaborative reinforcement learning
• global performance
• inter-agent collaboration

Innovation

• Collaborative Reinforcement Learning
• Multi-Agent Optimization
• Personalized Memory System
• Markov Decision Process
• Credit Assignment