Problem
Research questions and friction points this paper is trying to address.
inference routing
agent orchestration
declarative policy
cross-layer verification
policy drift
From Inference Routing to Agent Orchestration: Declarative Policy Compilation with Cross-Layer Verification
🤖 AI Summary
This work addresses strategy inconsistency and drift arising from cross-team collaboration in large model inference routing and multi-step agent workflows. To this end, the authors propose a non-Turing-complete declarative domain-specific language (DSL) that uniformly specifies end-to-end policies spanning inference gateways, agent orchestration, and infrastructure deployment. For the first time, declarative policy specification is extended to multi-step agent orchestration, with a compiler backend that generates artifacts for diverse targets—including LangGraph, Kubernetes, YANG/NETCONF, and MCP/A2A—while incorporating built-in formal verification. The approach enables one-click propagation of policy changes across the full stack, ensuring cross-layer consistency, conflict-free execution, and structured auditability, thereby substantially reducing manual coordination overhead.
📝 Abstract
The Semantic Router DSL is a non-Turing-complete policy language deployed in production for per-request LLM inference routing: content signals (embedding similarity, PII detection, jailbreak scoring) feed into weighted projections and priority-ordered decision trees that select a model, enforce privacy policies, and produce structured audit traces -- all from a single declarative source file. Prior work established conflict-free compilation for probabilistic predicates and positioned the DSL within the Workload-Router-Pool inference architecture.
This paper extends the same language from stateless, per-request routing to multi-step agent workflows -- the full path from inference gateway to agent orchestration to infrastructure deployment. The DSL compiler emits verified decision nodes for orchestration frameworks (LangGraph, OpenClaw), Kubernetes artifacts (NetworkPolicy, Sandbox CRD, ConfigMap), YANG/NETCONF payloads, and protocol-boundary gates (MCP, A2A) -- all from the same source.
Because the language is non-Turing-complete, the compiler guarantees exhaustive routing, conflict-free branching, referential integrity, and audit traces structurally coupled to the decision logic. Because signal definitions are shared across targets, a threshold change propagates from inference gateway to agent gate to infrastructure artifact in one compilation step -- eliminating cross-team coordination as the primary source of policy drift. We ground the approach in four pillars -- auditability, cost efficiency, verifiability, and tunability -- and identify the verification boundary at each layer.
Innovation
Methods, ideas, or system contributions that make the work stand out.
declarative policy compilation
agent orchestration
cross-layer verification
non-Turing-complete DSL
policy drift elimination
🔎 Similar Papers
No similar papers found.
