Supporting Dynamic Agentic Workloads: How Data and Agents Interact

2025-12-10

AI Summary

Existing data management architectures struggle to support LLM-driven multi-agent systems—whose dynamic task decomposition, cross-modal attention shifts, and collaborative intermediate-result sharing induce highly nondeterministic, multimodal, and massively concurrent workloads that severely challenge conventional query optimization and caching. This paper proposes Agent-Centric Data Weaving, a novel architecture featuring attention-guided retrieval, semantic micro-caching, predictive prefetching, and quorum-based data serving—enabling real-time perception of agent behavior and coordinated data-system optimization. Integrating LLM-powered semantic understanding, context-aware query optimization, distributed coordination, and agent behavioral modeling, the architecture significantly reduces redundant queries and data migration overhead, improves intermediate-result sharing efficiency, and lowers overall inference latency. It provides an adaptive data infrastructure for dynamic, high-concurrency, nondeterministic agent workflows.

Technology Category

• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Multiagent Systems: Agent/AI Theories and Architectures
• Search and Optimization: Distributed Search

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Agentic search
• Economics, Online Markets and Human Computation: Architectures and workflows that use LLMs for crowd work

Abstract

The rise of multi-agent systems powered by large language models (LLMs) and specialized reasoning agents exposes fundamental limitations in today's data management architectures. Traditional databases and data fabrics were designed for static, well-defined workloads, whereas agentic systems exhibit dynamic, context-driven, and collaborative behaviors. Agents continuously decompose tasks, shift attention across modalities, and share intermediate results with peers - producing non-deterministic, multi-modal workloads that strain conventional query optimizers and caching mechanisms. We propose an Agent-Centric Data Fabric, a unified architecture that rethinks how data systems serve, optimize, coordinate, and learn from agentic workloads. To achieve this we exploit the concepts of attention-guided data retrieval, semantic micro-caching for context-driven agent federations, predictive data prefetching and quorum-based data serving. Together, these mechanisms enable agents to access representative data faster and more efficiently, while reducing redundant queries, data movement, and inference load across systems. By framing data systems as adaptive collaborators, instead of static executors, we outline new research directions toward behaviorally responsive data infrastructures, where caching, probing, and orchestration jointly enable efficient, context-rich data exchange among dynamic, reasoning-driven agents.

Problem

• Addresses dynamic data needs of multi-agent systems
• Proposes an Agent-Centric Data Fabric for adaptive collaboration
• Enables efficient data exchange among reasoning-driven agents

Innovation

• Agent-Centric Data Fabric architecture
• Attention-guided retrieval and semantic micro-caching
• Predictive prefetching with quorum-based data serving