Relay Selection in Wireless Networks as Restless Bandits

2025-08-28

AI Summary

This paper addresses the slot-level relay selection problem in wireless networks where direct source-destination links are blocked, necessitating multi-hop forwarding of large files via candidate relays, with the objective of minimizing the long-term average packet holding cost at relays. Method: We first establish that this problem can be formulated as a Restless Multi-Armed Bandit (RMAB) and prove its Whittle indexability. Leveraging this property, we propose an efficient algorithm to compute the Whittle index for each relay in closed form and design a low-complexity, index-based online scheduling policy. Contribution/Results: Theoretical analysis guarantees the asymptotic optimality of the proposed policy. Simulation results demonstrate significant improvements over state-of-the-art methods in terms of average packet holding cost, end-to-end delay, and throughput, thereby validating the effectiveness and practicality of Whittle-index-based control for dynamic wireless relay networks.

Technology Category

• Planning, Routing, and Scheduling: Scheduling under Uncertainty
• Machine Learning: Online Learning & Bandits
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Web Mining and Content Analysis: Models for Web evolution

Abstract

We consider a wireless network in which a source node needs to transmit a large file to a destination node. The direct wireless link between the source and the destination is assumed to be blocked. Multiple candidate relays are available to forward packets from the source to the destination. A holding cost is incurred for each packet stored at every relay in each time slot. The objective is to design a policy for selecting a relay in each time slot to which the source attempts to send a packet, so as to minimize the expected long-run time-averaged total packet holding cost at the relays. This problem is an instance of the restless multi-armed bandit (RMAB) problem, which is provably hard to solve. We prove that this relay selection problem is Whittle-indexable, and propose a method to compute the Whittle index of each relay in every time slot. In each time slot, our relay selection policy transmits a packet to the relay with the smallest Whittle index. Using simulations, we show that the proposed policy outperforms the relay selection policies proposed in prior work in terms of average cost, delay, as well as throughput.

Problem

• Selecting optimal relays to minimize packet holding costs
• Designing Whittle-index policy for restless multi-armed bandits
• Improving wireless network performance through efficient relay selection

Innovation

• Modeling relay selection as restless bandits problem
• Proposing Whittle-indexable solution for relay selection
• Selecting relay with smallest Whittle index each slot