ContextWeaver: Selective and Dependency-Structured Memory Construction for LLM Agents

2026-04-24

AI Summary

This work addresses the challenge that large language model (LLM) agents struggle to effectively retain and utilize early critical structured information during long-context interactions, often neglecting the causal and logical dependencies essential for multi-step reasoning. To overcome this limitation, the authors propose a dependency-driven memory construction framework that models interaction trajectories as reasoning-step graphs. The approach dynamically extracts relevant context through root-to-node path compression and integrates execution feedback for lightweight verification. By combining graph-structured dependency modeling, path-based summarization, and retrieval augmentation, the method significantly outperforms sliding-window baselines on the SWE-Bench Verified and Lite benchmarks, achieving higher pass@1 accuracy while reducing both reasoning steps and token consumption.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Large language model (LLM) agents often struggle in long-context interactions. As the agent accumulates more interaction history, context management approaches such as sliding window and prompt compression may omit earlier structured information that later steps rely on. Recent retrieval-based memory systems surface relevant content but still overlook the causal and logical structure needed for multi-step reasoning. We introduce ContextWeaver, a selective and dependency-structured memory framework that organizes an agent's interaction trace into a graph of reasoning steps and selects the relevant context for future actions. Unlike prior context management approaches, ContextWeaver supports: (1) dependency-based construction and traversal that link each step to the earlier steps it relies on; (2) compact dependency summarization that condenses root-to-step reasoning paths into reusable units; and (3) a lightweight validation layer that incorporates execution feedback. On the SWE-Bench Verified and Lite benchmarks, ContextWeaver improves performance over a sliding-window baseline in pass@1, while reducing reasoning steps and token usage. Our observations suggest that modeling logical dependencies provides a stable and scalable memory mechanism for LLM agents that use tools.

Problem

• long-context interaction
• memory management
• logical dependency
• multi-step reasoning
• LLM agents

Innovation

• dependency-structured memory
• reasoning graph
• context summarization
• LLM agents
• execution feedback