Problem
Research questions and friction points this paper is trying to address.
AI-driven virtual worlds
agent orchestration
heterogeneous AI backends
edge-cloud coordination
service routing
From Prompt to Service: An SLM-Based Agent Orchestration Gateway for AI-Driven Virtual Worlds
🤖 AI Summary
This work addresses the challenges of coordinating multimodal user requests with heterogeneous AI backends in AI-driven virtual worlds, where high system coupling and poor scalability hinder extensibility. To overcome these limitations, we propose an intelligent agent orchestration gateway based on small language models (SLMs). The gateway leverages a lightweight SLM at the edge for semantic intent recognition and employs a configurable service registry to enable intent-driven dynamic routing, thereby decoupling clients from diverse AI backends and facilitating seamless integration of new capabilities. Our hierarchical architecture—combining a lightweight routing model with a large model for response generation—was validated on the InterwovenXR virtual museum platform. Results demonstrate that sub-billion-parameter SLMs can perform intent routing efficiently with low latency, and the overall solution is deployable on mid-tier edge devices, outperforming single-model architectures in both flexibility and performance.
📝 Abstract
As generative AI capabilities expand, AI-driven virtual worlds face a growing architectural challenge. Users interact through in-world interfaces in multimodal ways, yet their requests demand fundamentally different AI backend models and computational resources. Embedding these capabilities directly into virtual world systems reduces extensibility, complicates maintenance, and limits the ability to coordinate services distributed across edge and cloud infrastructure. This paper presents an SLM-based Agent Orchestration Gateway, a lightweight runtime coordination mechanism that decouples a virtual world client from heterogeneous AI backends through intent-driven service routing. An edge-deployed SLM classifies the semantic intent of each user prompt, a configurable service registry validates and resolves the routing decision, and the selected backend is invoked transparently, enabling new AI capabilities to be introduced in the virtual world without modifying the client application. The gateway is implemented and evaluated within the InterwovenXR virtual museum testbed. The evaluation shows that compact SLMs can serve as reliable intent routers on edge hardware, and that task-specific fine-tuning can transform sub-billion-parameter models into practical, low-latency routers. A layered configuration pairing a fine-tuned sub billion-parameter model as router with a larger SLM for conversational response generation is shown to be deployable on mid-range edge hardware and more efficient than delegating both responsibilities to a single model. The findings show that SLMs can support practical AI service orchestration in virtual worlds and the work contributes an evaluated architecture for scalable, extensible, and edge-supported AI interaction, enabling virtual agents become access points to distributed generative AI services.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Small Language Models (SLMs)
Agent Orchestration
Intent-Driven Routing
Edge AI
Virtual Worlds
🔎 Similar Papers
