Honest Lying: Understanding Memory Confabulation in Reflexive Agents

2026-05-28

AI Summary

This work addresses the tendency of reflective agents to generate confident yet erroneous self-diagnoses following task failure, leading to performance stagnation due to repeated reliance on such fabricated memories. The study formally defines and quantifies the “memory hallucination” problem, introducing a log-driven Reflective Repetition Rate (RRR) metric to detect systematic faulty reflections. To break this cycle of false beliefs, the authors replace open-ended self-diagnosis with procedural, trajectory-level failure signals. Experimental results in ALFWorld demonstrate that this approach increases the correct goal mention rate from 0% to 86%, reduces RRR from 0.64 to 0.10, and successfully resolves three previously unsolvable environments, substantially enhancing agent robustness and learning efficiency.

Abstract

Reflexion-style agents rely on self-generated reflections as memory, implicitly assuming that agents can accurately diagnose their own failures.We show that this assumption can fail systematically: across ALFWorld and HumanEval, agents store confident but incorrect interpretations of the task and continue acting on them across trials,even though the environment resets to the correct task each time. We call this failure mode memory confabulation and introduce the Reflection Repetition Rate (RRR), a log-based metric that detects repeated reliance on incorrect reflective content.Using RRR, we identify 16 frozen environments in ALFWorld, where 0 of 121 reflections mention the correct target object, and 4 analogous cases in HumanEval. Our mitigation replaces open-ended self-diagnosis with programmatic extraction of trajectory-level failure signals, increasing correct object mention from 0% to 86%, reducing RRR from 0.64 to 0.10, and solving 3 of 16 frozen ALFWorld environments, suggesting that reflective memory can reinforce false beliefs rather than correct them.

Problem

• memory confabulation
• reflexive agents
• self-diagnosis
• reflection repetition
• task misinterpretation

Innovation

• memory confabulation
• Reflexion agents
• Reflection Repetition Rate
• failure signal extraction
• self-diagnosis