MinGRU-Based Encoder for Turbo Autoencoder Frameworks

2025-03-11

AI Summary

To address scalability and training inefficiency bottlenecks in sequence modeling for long-code-length neural channel coding, this paper proposes a parallel Turbo autoencoder architecture integrating lightweight minGRU with the Mamba-derived Structured State Space Model (S6-SSM). This work is the first to jointly embed minGRU and Mamba modules within the TurboAE framework, preserving the strong sequential modeling capability of RNNs while leveraging the linear-time complexity of SSMs. Experiments demonstrate that, under long block lengths, the proposed architecture achieves over 40% faster training and approximately 35% lower memory consumption, while matching the block error rate (BLER) performance of CNN-based baselines. The results validate the feasibility and practicality of an efficient RNN–SSM hybrid paradigm for neural coding, establishing a novel architectural paradigm for long-sequence communication systems that jointly optimizes performance and computational efficiency.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Machine Learning: Deep Generative Models & Autoencoders
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Early neural channel coding approaches leveraged dense neural networks with one-hot encodings to design adaptive encoder-decoder pairs, improving block error rate (BLER) and automating the design process. However, these methods struggled with scalability as the size of message sets and block lengths increased. TurboAE addressed this challenge by focusing on bit-sequence inputs rather than symbol-level representations, transforming the scalability issue associated with large message sets into a sequence modeling problem. While recurrent neural networks (RNNs) were a natural fit for sequence processing, their reliance on sequential computations made them computationally expensive and inefficient for long sequences. As a result, TurboAE adopted convolutional network blocks, which were faster to train and more scalable, but lacked the sequential modeling advantages of RNNs. Recent advances in efficient RNN architectures, such as minGRU and minLSTM, and structured state space models (SSMs) like S4 and S6, overcome these limitations by significantly reducing memory and computational overhead. These models enable scalable sequence processing, making RNNs competitive for long-sequence tasks. In this work, we revisit RNNs for Turbo autoencoders by integrating the lightweight minGRU model with a Mamba block from SSMs into a parallel Turbo autoencoder framework. Our results demonstrate that this hybrid design matches the performance of convolutional network-based Turbo autoencoder approaches for short sequences while significantly improving scalability and training efficiency for long block lengths. This highlights the potential of efficient RNNs in advancing neural channel coding for long-sequence scenarios.

Problem

• Address scalability issues in neural channel coding for large message sets.
• Improve training efficiency and performance for long block lengths.
• Integrate lightweight RNNs with SSMs for scalable sequence processing.

Innovation

• Integrates minGRU with Mamba block
• Enables scalable sequence processing
• Improves training efficiency for long sequences