Online LLM Selection via Constrained Bandits with Time-Varying Demand

2026-06-15

AI Summary

This work addresses the online selection of heterogeneous large language models (LLMs) in edge-cloud inference systems under time-varying task demands, hard resource budgets (e.g., cost), and soft quality-of-service constraints (e.g., latency). It formulates this problem for the first time as a constrained stochastic multi-armed bandit. The authors propose an online learning algorithm that integrates upper confidence bound estimation, demand prediction, and constraint handling to balance reward maximization with long-term constraint satisfaction, operating under partial feedback and unknown performance distributions. Theoretical analysis establishes sublinear regret and soft constraint violation bounds, while experiments demonstrate the method’s effectiveness and robustness in dynamic, resource-constrained environments.

Abstract

Large Language Models (LLMs) are increasingly deployed in edge-cloud inference systems to handle diverse user tasks with heterogeneous accuracy, latency, and cost profiles. Selecting the appropriate LLM for each incoming task is critical for ensuring service quality and efficient resource utilization. However, model heterogeneity, stochastic and unknown performance characteristics, and time-varying task demands make static selection strategies inadequate. Real-world deployments often impose hard resource budgets such as monetary expenditure limits, along with soft service-level requirements such as latency guarantees. These constraints introduce additional challenges for online decision-making. We formulate this problem as a constrained stochastic bandit learning task, where the learner sequentially selects models under both packing-type (hard) and covering-type (soft) constraints, while adapting to time-varying task demand. The learner operates without access to the underlying reward, cost, or latency distributions and must rely on partial feedback. We develop a novel online learning algorithm that leverages confidence-bound estimates and demand predictions to balance reward maximization with long-term constraint satisfaction. We provide theoretical guarantees showing sublinear regret and sublinear covering constraint violations compared to an offline benchmark with full information. Experimental results on synthetic workloads demonstrate the effectiveness and robustness of our approach in dynamic, resource-constrained environments.

Problem

• Large Language Models
• Online Selection
• Constrained Bandits
• Time-Varying Demand
• Resource Constraints

Innovation

• Constrained Bandits
• Time-Varying Demand
• Online LLM Selection
• Confidence-Bound Estimation
• Resource-Constrained Inference