Software-Defined Agentic Serving
🤖 AI Summary
This work addresses the limitations of existing large language model–based multi-agent systems in adapting to dynamic environments and flexibly controlling communication properties. Inspired by Software-Defined Networking (SDN), the study introduces, for the first time, software-defined principles into agent services, proposing a runtime state-aware and programmable communication control framework. This framework enables dynamic, intent-driven, and responsive service orchestration based on high-level objectives, facilitating efficient system-wide coordination. Experimental results demonstrate that the proposed approach significantly enhances both the responsiveness and service efficiency of multi-agent systems, laying a foundational step toward realizing intent-driven, advanced multi-agent services.
Application Category
📝 Abstract
As multi-agent LLM pipelines grow in complexity, existing serving paradigms fail to adapt to the dynamic serving conditions. We argue that agentic serving systems should be programmable and system-aware, unlike existing serving which statically encode the parameters. In this work, we propose a new SDN-inspired agentic serving framework that helps control the key attributes of communication based on runtime state. This architecture enables serving-efficient, responsive agent systems and paves the way for high-level intent-driven agentic serving.
Problem
Research questions and friction points this paper is trying to address.
agentic serving
dynamic serving conditions
system-aware
programmable serving
multi-agent LLM pipelines
Innovation
Methods, ideas, or system contributions that make the work stand out.
Software-Defined Agentic Serving
multi-agent LLM pipelines
system-aware serving
runtime state adaptation
intent-driven serving
🔎 Similar Papers
No similar papers found.
