SpectrumFM: A Foundation Model for Intelligent Spectrum Management

2025-05-02

arXiv.org

AI Summary

To address the low recognition accuracy, slow convergence, and poor generalization of existing small-scale models in dynamic spectrum environments, this paper proposes SpectrumFM—a spectral foundation model. Methodologically, SpectrumFM integrates CNNs with multi-head self-attention to enhance IQ-signal representation learning; introduces the first foundation-model paradigm for spectrum analysis, featuring dual self-supervised pretraining tasks—masked signal reconstruction and next-time-step signal prediction; and employs parameter-efficient fine-tuning (e.g., LoRA) for cross-task transfer. Experiments demonstrate significant improvements: 12.1% higher accuracy in automatic modulation classification (AMC), 9.3% gain in wireless technology classification (WTC), an AUC of 0.97 for spectrum sensing at −4 dB SNR, over 10% improvement in anomaly detection performance, faster convergence, and markedly enhanced few-shot adaptation capability.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Other Foundations of Cognitive Modeling & Systems
• Computer Vision: Multi-modal Vision

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Intelligent spectrum management is crucial for improving spectrum efficiency and achieving secure utilization of spectrum resources. However, existing intelligent spectrum management methods, typically based on small-scale models, suffer from notable limitations in recognition accuracy, convergence speed, and generalization, particularly in the complex and dynamic spectrum environments. To address these challenges, this paper proposes a novel spectrum foundation model, termed SpectrumFM, establishing a new paradigm for spectrum management. SpectrumFM features an innovative encoder architecture that synergistically exploits the convolutional neural networks and the multi-head self-attention mechanisms to enhance feature extraction and enable robust representation learning. The model is pre-trained via two novel self-supervised learning tasks, namely masked reconstruction and next-slot signal prediction, which leverage large-scale in-phase and quadrature (IQ) data to achieve comprehensive and transferable spectrum representations. Furthermore, a parameter-efficient fine-tuning strategy is proposed to enable SpectrumFM to adapt to various downstream spectrum management tasks, including automatic modulation classification (AMC), wireless technology classification (WTC), spectrum sensing (SS), and anomaly detection (AD). Extensive experiments demonstrate that SpectrumFM achieves superior performance in terms of accuracy, robustness, adaptability, few-shot learning efficiency, and convergence speed, consistently outperforming conventional methods across multiple benchmarks. Specifically, SpectrumFM improves AMC accuracy by up to 12.1% and WTC accuracy by 9.3%, achieves an area under the curve (AUC) of 0.97 in SS at -4 dB signal-to-noise ratio (SNR), and enhances AD performance by over 10%.

Problem

• Develops a foundation model to improve spectrum management accuracy and generalization.
• Addresses limitations in existing methods for complex, dynamic spectrum environments.
• Enhances performance in tasks like modulation classification and anomaly detection.

Innovation

• CNN and self-attention encoder for robust feature extraction
• Self-supervised pre-training with masked and next-slot prediction tasks
• Parameter-efficient fine-tuning adapts to diverse spectrum management tasks