Beam Scheduling in Millimeter Wave Networks Using the Whittle Index

2025-03-23

AI Summary

This paper addresses dynamic user scheduling in a single-cell millimeter-wave (mmWave) downlink under constrained base station beam resources, aiming to minimize the long-term average cost—comprising queue holding and beam activation costs. We formulate the problem as a Restless Multi-Armed Bandit (RMAB) and, for the first time, rigorously prove its Whittle indexability. Leveraging this property, we derive a closed-form, analytically tractable, and computationally efficient expression for the Whittle index, and design a real-time scheduling policy based on the “lowest-index-first” principle. Our approach jointly incorporates mmWave channel state awareness and stochastic queue analysis. Simulation results demonstrate that the proposed method reduces average cost by over 30%, decreases end-to-end latency by more than 25%, and significantly improves energy efficiency compared to state-of-the-art baselines. Key contributions include: (i) the first theoretical proof of Whittle indexability for mmWave beam scheduling; (ii) an efficient closed-form Whittle index computation; and (iii) a low-complexity online scheduling policy with provable performance guarantees.

Technology Category

• Planning, Routing, and Scheduling: Scheduling under Uncertainty
• Machine Learning: Online Learning & Bandits
• Multiagent Systems: Mechanism Design

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Energy management for devices in mobile Web and WoT environments
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

We address the problem of beam scheduling for downlink transmissions in a single-cell millimeter wave (mmWave) network. The cell contains a mmWave base station (mBS) and its associated users. At the end of each time slot, a packet arrives into the queue of a user at the mBS with a certain probability. A holding cost is incurred for the packets stored in a user's queue at the mBS in every time slot. The number of simultaneous beams that the mBS can form to different users is less than the number of associated users. Also, a cost is incurred whenever a beam is formed from the mBS to a user. In a given time slot, a packet transmitted from the mBS to a user that has been assigned a beam is successfully received (respectively, not received) if the channel quality between the mBS and the user is good (respectively, bad). In every time slot, the mBS needs to assign the available beams to a subset of the users, in order to minimize the long-run expected average cost. This problem can be modeled as a restless multi-armed bandit problem, which is provably hard to solve. We prove the Whittle indexability of the above beam scheduling problem and propose a strategy to compute the Whittle index of each user. In each time slot, our proposed beam scheduling policy assigns beams to the users with the smallest Whittle indices. Using extensive simulations, we show that our proposed Whittle index-based beam scheduling policy significantly outperforms several scheduling policies proposed in prior work in terms of the average cost, average delay, as well as energy efficiency.

Problem

• Optimize beam scheduling in mmWave networks to minimize cost
• Address limited simultaneous beams versus user count challenge
• Prove Whittle indexability and design efficient scheduling policy

Innovation

• Uses Whittle index for beam scheduling
• Minimizes long-run expected average cost
• Outperforms prior policies in simulations