MemSyco-Bench: Benchmarking Sycophancy in Agent Memory

2026-07-01

AI Summary

Existing agent memory benchmarks overlook the impact of memory retrieval on downstream reasoning and notably lack evaluation of “memory-induced sycophancy”—a phenomenon where agents compromise factual accuracy to align with user preferences. This work systematically incorporates this critical issue into benchmarking by introducing MemSyco-Bench, a comprehensive evaluation framework encompassing five core capabilities: rejecting the misuse of memory as fact, respecting the contextual boundaries of memory applicability, reconciling conflicts between memory and objective evidence, tracking memory updates, and effectively leveraging memory for personalization. Built upon large language model–based agent architectures and evaluated through structured tasks across diverse scenarios, MemSyco-Bench enables quantitative assessment of the reasonableness and reliability of memory usage. The benchmark is open-sourced to provide the research community with a unified tool for advancing the objectivity and robustness of agent memory systems.

Abstract

Memory has emerged as a cornerstone of modern LLM-based agents, supporting their evolution from single-turn assistants to long-term collaborators. However, memory is not always beneficial: retrieved memories often induce a critical issue of sycophancy, causing agents to over-align with the user at the cost of factual accuracy or objective reasoning. Despite this emerging risk, existing memory benchmarks primarily evaluate whether memories are correctly stored, retrieved, or updated, while overlooking how retrieved memories influence downstream reasoning and decision-making. To bridge this gap, we propose MemSyco-Bench, a comprehensive benchmark for evaluating memory-induced sycophancy in agent systems. MemSyco-Bench measures when memory should influence a decision and how valid memory should be used. Specifically, it covers five tasks that assess whether agents can reject memory as factual evidence, respect its applicable scope, resolve conflicts between memory and objective evidence, track memory updates, and use valid memory for personalization. All related resources are collected for the community at https://github.com/XMUDeepLIT/MemSyco-Bench.

Problem

• sycophancy
• agent memory
• memory-induced bias
• LLM-based agents
• reasoning integrity

Innovation

• sycophancy
• agent memory
• memory-induced reasoning
• LLM-based agents
• benchmarking