Hierarchical Reinforcement Learning for Next Generation of Multi-AP Coordinated Spatial Reuse

2026-03-21

AI Summary

This work addresses the challenge in multi-access point (AP) coordinated spatial reuse (C-SR), where high coordination overhead and complex parameter optimization hinder the simultaneous achievement of high throughput and fairness. To this end, the paper proposes a two-layer multi-armed bandit (MAB) algorithm that, for the first time, introduces hierarchical reinforcement learning into the multi-AP C-SR setting. The proposed framework jointly optimizes scheduling, power control, and link adaptation through a hierarchical structure, significantly reducing coordination overhead while meeting diverse quality-of-service (QoS) requirements. System-level simulations demonstrate that the scheme not only enhances aggregate network throughput but also improves fairness in resource allocation among users, offering a robust and efficient solution for next-generation Wi-Fi systems.

Technology Category

• Multiagent Systems: Coordination and Collaboration
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Intelligent Robots: Multi-Robot Systems

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs

Abstract

In next generation of Wi-Fi networks Multiple Access Point Coordination (MAPC) is poised to significantly enhance the network performance by enabling a set of Access Points (APs) to coordinate with each other through advanced coordinating schemes so that to reduce inter-AP contention and congestion. This paper focuses on defining a framework to facilitate the coordination across multi-APs when these employ Coordinated Spatial Reuse (C-SR). In this case, the coordinating APs may need to reciprocally adjust their scheduling strategy, power control and link adaptation to meet specific Quality of Service (QoS) requirements, which by using classical approaches leads to high overhead due to negotiations needed across APs, and requires complex solutions in order to properly optimize the network across all the parameters in play. In this matter, a two layer Multi-Armed Bandit (MAB) algorithm has been proposed to optimize such a network while preserving the fair use of resources across all nodes. The validity of this holistic approach is confirmed by system level simulations, which show that the proposed algorithm not only improves the network in terms of sum-throughput, but also allows to enhance fairness, making this a robust solution for next-generation of Wi-Fi networks.

Problem

• Multi-AP Coordination
• Coordinated Spatial Reuse
• Quality of Service
• Wi-Fi Networks
• Resource Fairness

Innovation

• Hierarchical Reinforcement Learning
• Multi-AP Coordination
• Coordinated Spatial Reuse
• Multi-Armed Bandit
• Fairness-aware Optimization