End-to-end Learning of Probabilistic and Geometric Constellation Shaping with Iterative Receivers

2025-10-26

AI Summary

This work addresses the performance limitation of constellation modulation imposed by fixed symbol probability distributions and geometric structures. We propose a novel end-to-end joint optimization framework for probabilistic and geometric shaping. Specifically, an auxiliary shaping encoder is introduced to model non-uniform symbol priors, while constellation shaping is embedded into the full iterative detection-and-decoding loop. For the first time, deep unfolding is employed to enable differentiable joint learning across the entire physical-layer pipeline. Under block-fading channels, the proposed scheme significantly outperforms conventional APSK and QAM: it achieves 0.3 dB and 0.15 dB BER gains over APSK under two receiver configurations, with the iterative variant further improving performance by 0.1 dB relative to standard APSK. The key contribution lies in the first application of deep unfolding to joint constellation shaping within a full-chain iterative receiver architecture, enabling synergistic optimization across both probabilistic and geometric dimensions.

Technology Category

• Search and Optimization: Learning to Search
• Reasoning under Uncertainty: Stochastic Optimization
• Planning, Routing, and Scheduling: Scheduling under Uncertainty

Application Category

• Security and Privacy: Privacy-enhancing technologies
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

An end-to-end learning method for constellation shaping with a shaping-encoder assisted transceiver architecture is presented. The shaping encoder, which produces shaping bits with a higher probability of zeros, is used to produce an efficient symbol probability distribution. Both the probability distribution and the constellation geometry are jointly optimized, using end-to-end learning. Optimized constellations are evaluated using two iterative receiver architectures. Bit error rate (BER) performance gain is quantified against standard amplitude phase-shift keying (APSK) and quadrature amplitude modulation (QAM) constellations. A maximum BER gain of 0.3 dB and 0.15 dB are observed under two receivers for the learned constellations compared to standard APSK or QAM. The basic approach is extended to incorporate the full iterative detection and decoding loop, using the deep unfolding technique. A bit error rate gain of 0.1 dB is observed for the iterative scheme with learned constellations under block fading channel conditions, when compared to standard APSK.

Problem

• Jointly optimizing probability distribution and constellation geometry for communication systems
• Evaluating optimized constellations with iterative receiver architectures for performance gain
• Extending approach to incorporate full iterative detection and decoding loop

Innovation

• End-to-end learning optimizes constellation shaping
• Joint probability and geometry optimization technique
• Deep unfolding enables iterative detection and decoding