Multi-Agent Actor-Critic with Harmonic Annealing Pruning for Dynamic Spectrum Access Systems

2025-03-19

AI Summary

To address the deployment challenges of multi-agent deep reinforcement learning (MADRL) for dynamic spectrum access (DSA) on resource-constrained edge devices, this paper proposes a progressive structured pruning method tailored to the independent-actor-global-critic recurrent MARL framework. Our key contributions are: (1) a harmonic annealing sparsity scheduler that significantly outperforms linear and polynomial scheduling under high pruning rates; and (2) the first integration of progressive pruning into the recurrent MARL training pipeline, enabling joint optimization of model compactness and policy performance. Experiments demonstrate that our method consistently surpasses conventional DSA approaches, mainstream MADRL baselines, and state-of-the-art pruning techniques across diverse training settings. Remarkably, at sparsity levels exceeding 90%, policy performance improves rather than degrades—enabling decentralized, low-overhead, real-time spectrum decision-making on edge hardware.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Multiagent Systems: Adversarial Agents
• Planning, Routing, and Scheduling: Planning with Markov Models (MDPs, POMDPs)

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Multi-Agent Deep Reinforcement Learning (MADRL) has emerged as a powerful tool for optimizing decentralized decision-making systems in complex settings, such as Dynamic Spectrum Access (DSA). However, deploying deep learning models on resource-constrained edge devices remains challenging due to their high computational cost. To address this challenge, in this paper, we present a novel sparse recurrent MARL framework integrating gradual neural network pruning into the independent actor global critic paradigm. Additionally, we introduce a harmonic annealing sparsity scheduler, which achieves comparable, and in certain cases superior, performance to standard linear and polynomial pruning schedulers at large sparsities. Our experimental investigation demonstrates that the proposed DSA framework can discover superior policies, under diverse training conditions, outperforming conventional DSA, MADRL baselines, and state-of-the-art pruning techniques.

Problem

• Optimizing decentralized decision-making in Dynamic Spectrum Access systems.
• Reducing computational cost for deep learning on edge devices.
• Improving performance with harmonic annealing sparsity scheduler.

Innovation

• Sparse recurrent MARL framework for DSA
• Harmonic annealing sparsity scheduler introduced
• Gradual neural network pruning integrated