Bandit-Based Rate Adaptation for a Single-Server Queue

2025-12-12

AI Summary

We study adaptive rate control for a single-queue system with unknown channel capacity and arrival process, and only binary ACK/NACK feedback—without prior knowledge of the slack ε (i.e., the gap between service and arrival rates)—and aim to guarantee bounded time-average expected queue length. Method: We propose a multi-phase progressive discretization algorithm integrating Upper Confidence Bound (UCB) exploration, stochastic queuing analysis, and information-theoretic lower-bound construction. Contribution/Results: Our approach achieves an O(log^{3.5}(1/ε)/ε³) upper bound on the time-average expected queue length under fully unknown ε—first such result without ε knowledge. When ε is known, a single-phase UCB policy attains O(log(1/ε)/ε²), tight up to logarithmic factors. We further establish a fundamental Ω(1/ε²) lower bound, revealing ε’s intrinsic role as a bottleneck. This work breaks the classical requirement of prior ε knowledge and provides theoretically optimal guarantees for adaptive rate control under partial feedback.

Technology Category

• Reasoning under Uncertainty: Sequential Decision Making
• Machine Learning: Online Learning & Bandits
• Planning, Routing, and Scheduling: Scheduling under Uncertainty

Application Category

• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Security and Privacy: Large-scale security measurements

Abstract

This paper considers the problem of obtaining bounded time-average expected queue sizes in a single-queue system with a partial-feedback structure. Time is slotted; in slot $t$ the transmitter chooses a rate $V(t)$ from a continuous interval. Transmission succeeds if and only if $V(t)le C(t)$, where channel capacities ${C(t)}$ and arrivals are i.i.d. draws from fixed but unknown distributions. The transmitter observes only binary acknowledgments (ACK/NACK) indicating success or failure. Let $varepsilon>0$ denote a sufficiently small lower bound on the slack between the arrival rate and the capacity region. We propose a phased algorithm that progressively refines a discretization of the uncountable infinite rate space and, without knowledge of $varepsilon$, achieves a $mathcal{O}!ig(log^{3.5}(1/varepsilon)/varepsilon^{3}ig)$ time-average expected queue size uniformly over the horizon. We also prove a converse result showing that for any rate-selection algorithm, regardless of whether $varepsilon$ is known, there exists an environment in which the worst-case time-average expected queue size is $Ω(1/varepsilon^{2})$. Thus, while a gap remains in the setting without knowledge of $varepsilon$, we show that if $varepsilon$ is known, a simple single-stage UCB type policy with a fixed discretization of the rate space achieves $mathcal{O}!ig(log(1/varepsilon)/varepsilon^{2}ig)$, matching the converse up to logarithmic factors.

Problem

• Adapting transmission rates with partial feedback to minimize queue size
• Achieving bounded queue size without knowing slack between arrival and capacity
• Proving lower bound on queue size for any rate-selection algorithm

Innovation

• Phased algorithm progressively refines discretization of infinite rate space
• Achieves bounded queue size without knowledge of slack parameter epsilon
• UCB policy with fixed discretization matches lower bound when epsilon known