Transformer-based Scalable Beamforming Optimization via Deep Residual Learning

2025-10-14

AI Summary

To address the real-time optimization challenge of large-scale MU-MISO downlink beamforming under dynamic channels, this paper proposes a Transformer-based unsupervised learning-to-optimize (L2O) framework. Methodologically, it integrates residual iterative optimization, curriculum learning, semi-amortized training, and a sliding-window strategy to jointly model channel and beamforming features; it employs a multi-layer Transformer coupled with feed-forward networks, enabling offline training and millisecond-level online inference. Compared to state-of-the-art methods, the framework significantly outperforms baselines at low-to-moderate SNRs, approaches WMMSE performance at high SNRs, and accelerates inference by one to two orders of magnitude—achieving both high accuracy and real-time efficiency. This work establishes a scalable, deep-learning-driven paradigm for physical-layer communication design.

Technology Category

• Search and Optimization: Learning to Search
• Machine Learning: Optimization
• Computer Vision: Learning & Optimization for CV

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

We develop an unsupervised deep learning framework for downlink beamforming in large-scale MU-MISO channels. The model is trained offline, allowing real-time inference through lightweight feedforward computations in dynamic communication environments. Following the learning-to-optimize (L2O) paradigm, a multi-layer Transformer iteratively refines both channel and beamformer features via residual connections. To enhance training, three strategies are introduced: (i) curriculum learning (CL) to improve early-stage convergence and avoid local optima, (ii) semi-amortized learning to refine each Transformer block with a few gradient ascent steps, and (iii) sliding-window training to stabilize optimization by training only a subset of Transformer blocks at a time. Extensive simulations show that the proposed scheme outperforms existing baselines at low-to-medium SNRs and closely approaches WMMSE performance at high SNRs, while achieving substantially faster inference than iterative and online learning approaches.

Problem

• Optimizing downlink beamforming in large-scale MU-MISO channels
• Enabling real-time inference through offline-trained deep learning
• Improving beamformer performance via Transformer-based residual learning

Innovation

• Unsupervised deep learning framework for beamforming
• Transformer refines features via residual connections
• Curriculum and semi-amortized learning enhance training