Short Wins Long: Short Codes with Language Model Semantic Correction Outperform Long Codes

2025-05-13

AI Summary

To address high decoding latency and low semantic fidelity in natural language transmission over noisy wireless channels, this paper proposes a semantic communication framework integrating short block codes with large language models. Methodologically, sentences are segmented and independently encoded/decoded in parallel using short Polar codes; a semantic-enhanced decoder—incorporating a BART-based bidirectional Transformer—is designed for fragment-level semantic error correction; and a novel hybrid HARQ mechanism is introduced, leveraging semantic uncertainty modeling. The key contribution lies in breaking away from conventional long-code paradigms to achieve deep coupling between channel coding and semantic modeling. Experimental results demonstrate that, compared to LDPC-based long-code schemes, the proposed approach reduces block error rate (BLER) and semantic distortion significantly, cuts decoding latency by 3.2×, and improves semantic fidelity by 27.6%.

Technology Category

• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Large language models for search
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

This paper presents a novel semantic-enhanced decoding scheme for transmitting natural language sentences with multiple short block codes over noisy wireless channels. After ASCII source coding, the natural language sentence message is divided into segments, where each is encoded with short block channel codes independently before transmission. At the receiver, each short block of codewords is decoded in parallel, followed by a semantic error correction (SEC) model to reconstruct corrupted segments semantically. We design and train the SEC model based on Bidirectional and Auto-Regressive Transformers (BART). Simulations demonstrate that the proposed scheme can significantly outperform encoding the sentence with one conventional long LDPC code, in terms of block error rate (BLER), semantic metrics, and decoding latency. Finally, we proposed a semantic hybrid automatic repeat request (HARQ) scheme to further enhance the error performance, which selectively requests retransmission depends on semantic uncertainty.

Problem

• Transmitting natural language sentences over noisy wireless channels
• Semantic error correction for corrupted segments using BART
• Improving performance over long LDPC codes in BLER and latency

Innovation

• Short block codes with parallel decoding
• BART-based semantic error correction model
• Semantic HARQ for selective retransmission