A Novel Multiple Access Scheme for Heterogeneous Wireless Communications using Symmetry-aware Continual Deep Reinforcement Learning

2025-02-24

AI Summary

In metaverse-oriented heterogeneous wireless communications, dynamic spectrum multiple access faces challenges including non-stationary channels, coexistence of heterogeneous protocols, unknown numbers and types of user equipment, and requirements for backward compatibility and privacy preservation. This paper proposes a symmetry-aware continual deep reinforcement learning (DRL) medium access control (MAC) framework built upon the D3QL architecture, integrating group symmetry modeling, online policy optimization, and theoretically guaranteed convergence. We introduce the first lifelong evolutionary MAC policy learning mechanism, enabling seamless coordination with legacy devices under formal privacy constraints. Experimental results demonstrate that, compared to conventional DRL-based approaches, the proposed method achieves a 23.6% increase in system throughput, a 41.2% reduction in packet collision rate, a 35.8% improvement in Jain’s fairness index, and maintains sub-millisecond real-time responsiveness.

Technology Category

• Machine Learning: Life-Long and Continual Learning
• Multiagent Systems: Mechanism Design
• Search and Optimization: Learning to Search

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Responsible Web: Data and user privacy-enhancing technologies for the Web
• Security and Privacy: Large-scale security measurements

Abstract

The Metaverse holds the potential to revolutionize digital interactions through the establishment of a highly dynamic and immersive virtual realm over wireless communications systems, offering services such as massive twinning and telepresence. This landscape presents novel challenges, particularly efficient management of multiple access to the frequency spectrum, for which numerous adaptive Deep Reinforcement Learning (DRL) approaches have been explored. However, challenges persist in adapting agents to heterogeneous and non-stationary wireless environments. In this paper, we present a novel approach that leverages Continual Learning (CL) to enhance intelligent Medium Access Control (MAC) protocols, featuring an intelligent agent coexisting with legacy User Equipments (UEs) with varying numbers, protocols, and transmission profiles unknown to the agent for the sake of backward compatibility and privacy. We introduce an adaptive Double and Dueling Deep Q-Learning (D3QL)-based MAC protocol, enriched by a symmetry-aware CL mechanism, which maximizes intelligent agent throughput while ensuring fairness. Mathematical analysis validates the efficiency of our proposed scheme, showcasing superiority over conventional DRL-based techniques in terms of throughput, collision rate, and fairness, coupled with real-time responsiveness in highly dynamic scenarios.

Problem

• Efficient spectrum access in heterogeneous wireless environments
• Adapting DRL to non-stationary communication settings
• Enhancing MAC protocols with continual learning

Innovation

• Symmetry-aware Continual Learning
• Double and Dueling Deep Q-Learning
• Enhanced Medium Access Control