CLEAR: Context Augmentation from Contrastive Learning of Experience via Agentic Reflection

2026-04-08

AI Summary

This work addresses the challenge that large language model agents incur additional reasoning overhead when reusing historical experiences to adapt to new tasks, thereby increasing decision-making burden. To overcome this, the authors propose CLEAR, a novel framework that integrates contrastive learning with agent reflection for the first time. Specifically, a reflective agent performs contrastive analysis over past trajectories to generate task-relevant contextual summaries, which are then used to supervise the training of a Context-Augmented Model (CAM). The framework further incorporates reinforcement learning to refine CAM’s generation capabilities, effecting a paradigm shift from retrieval-based reuse to generative augmentation. Experiments demonstrate that CLEAR significantly outperforms strong baselines, improving task completion rates on the AppWorld benchmark from 72.62% to 81.15% and increasing average rewards on a WebShop subset from 0.68 to 0.74.

Abstract

Large language model agents rely on effective model context to obtain task-relevant information for decision-making. Many existing context engineering approaches primarily rely on the context generated from the past experience and retrieval mechanisms that reuse these context. However, retrieved context from past tasks must be adapted by the execution agent to fit new situations, placing additional reasoning burden on the underlying LLM. To address this limitation, we propose a generative context augmentation framework using Contrastive Learning of Experience via Agentic Reflection (CLEAR). CLEAR first employs a reflection agent to perform contrastive analysis over past execution trajectories and summarize useful context for each observed task. These summaries are then used as supervised fine-tuning data to train a context augmentation model (CAM). Then we further optimize CAM using reinforcement learning, where the reward signal is obtained by running the task execution agent. By learning to generate task-specific knowledge rather than retrieve knowledge from the past, CAM produces context that is better tailored to the current task. We conduct comprehensive evaluations on the AppWorld and WebShop benchmarks. Experimental results show that CLEAR consistently outperforms strong baselines. It improves task completion rate from 72.62% to 81.15% on AppWorld test set and averaged reward from 0.68 to 0.74 on a subset of WebShop, compared with baseline agent. Our code is publicly available at https://github.com/awslabs/CLEAR.

Problem

• context augmentation
• large language model agents
• task-relevant context
• retrieval mechanisms
• reasoning burden

Innovation

• context augmentation
• contrastive learning
• agentic reflection
• reinforcement learning
• large language model agents