Memento No More: Coaching AI Agents to Master Multiple Tasks via Hints Internalization

2025-02-03

AI Summary

Existing AI agents rely heavily on manual prompting and demonstration examples, limiting their ability to autonomously internalize multi-task knowledge, retain long-term memory, or generalize across tasks. Method: We propose “prompt internalization”—a context distillation mechanism that incrementally embeds external task knowledge directly into model weights via human-AI collaborative feedback, without requiring demonstration data. Built upon Llama-3, our agent architecture integrates multi-stage task orchestration (retrieval → tool invocation → question answering) with iterative weight updates. Contribution/Results: Experiments show that after only a few rounds of interactive feedback, our agent significantly outperforms GPT-4o and DeepSeek-V3 on composite task benchmarks, achieving higher task-sequence accuracy and markedly improved cross-task generalization stability. To our knowledge, this is the first work to realize end-to-end knowledge consolidation—from prompts to parameters—establishing a novel paradigm for self-learning AI agents.

Technology Category

• Multiagent Systems: Multiagent Learning
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Search and Optimization: Learning to Search

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Architectures and workflows that use LLMs for crowd work

Abstract

As the general capabilities of artificial intelligence (AI) agents continue to evolve, their ability to learn to master multiple complex tasks through experience remains a key challenge. Current LLM agents, particularly those based on proprietary language models, typically rely on prompts to incorporate knowledge about the target tasks. This approach does not allow the agent to internalize this information and instead relies on ever-expanding prompts to sustain its functionality in diverse scenarios. This resembles a system of notes used by a person affected by anterograde amnesia, the inability to form new memories. In this paper, we propose a novel method to train AI agents to incorporate knowledge and skills for multiple tasks without the need for either cumbersome note systems or prior high-quality demonstration data. Our approach employs an iterative process where the agent collects new experiences, receives corrective feedback from humans in the form of hints, and integrates this feedback into its weights via a context distillation training procedure. We demonstrate the efficacy of our approach by implementing it in a Llama-3-based agent which, after only a few rounds of feedback, outperforms advanced models GPT-4o and DeepSeek-V3 in a taskset requiring correct sequencing of information retrieval, tool use, and question answering.

Problem

• Autonomous Learning
• Complex Task Acquisition
• Language Models Limitations

Innovation

• Prompt Understanding
• Multi-task Learning
• Efficient Training Method