Learning from Supervision with Semantic and Episodic Memory: A Reflective Approach to Agent Adaptation

2025-10-22

AI Summary

This work addresses the challenge of enabling large language models (LLMs) to efficiently and transparently learn new classification tasks in a zero-shot, parameter-free setting. Methodologically, it introduces a memory-augmented zero-shot proxy framework featuring a dual-memory mechanism—integrating episodic memory (instance-level critiques) with semantic memory (task-level instructions)—augmented by retrieval, prompt engineering, memory abstraction, and evolutionary memory updating; crucially, it employs LLM-generated self-critiques to enable reflective adaptation. A novel metric, “inducibility,” is proposed to quantify an LLM’s responsiveness to supervision signals. Experiments demonstrate that the method achieves up to 24.8% absolute accuracy gain over a label-only RAG baseline across multiple classification tasks. Furthermore, it reveals systematic behavioral disparities between open- and closed-source LLMs in handling factual versus preference-laden data.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

We investigate how agents built on pretrained large language models can learn target classification functions from labeled examples without parameter updates. While conventional approaches like fine-tuning are often costly, inflexible, and opaque, we propose a memory-augmented framework that leverages both labeled data and LLM-generated critiques. Our framework uses episodic memory to store instance-level critiques-capturing specific past experiences-and semantic memory to distill these into reusable, task-level guidance. Across a diverse set of tasks, incorporating critiques yields up to a 24.8 percent accuracy improvement over retrieval-based (RAG-style) baselines that rely only on labels. Through extensive empirical evaluation, we uncover distinct behavioral differences between OpenAI and opensource models, particularly in how they handle fact-oriented versus preference-based data. To interpret how models respond to different representations of supervision encoded in memory, we introduce a novel metric, suggestibility. This helps explain observed behaviors and illuminates how model characteristics and memory strategies jointly shape learning dynamics. Our findings highlight the promise of memory-driven, reflective learning for building more adaptive and interpretable LLM agents.

Problem

• Enabling agents to learn classification functions without parameter updates
• Improving accuracy over retrieval-based baselines using memory critiques
• Analyzing model suggestibility to different supervision representations

Innovation

• Memory-augmented framework using labeled data and critiques
• Episodic memory stores instance-level past experiences
• Semantic memory distills experiences into reusable task guidance