LLM-Viterbi: Semantic-Aware Decoding for Convolutional Codes

2026-04-20

AI Summary

This work proposes a semantic-aware joint decoding framework that integrates a fine-tuned byte-level T5 (ByT5) large language model into the Viterbi decoding of convolutional codes over an AWGN channel. Departing from conventional bit-level channel coding that ignores textual semantics, the method maintains multiple candidate paths and periodically fuses channel reliability metrics with semantic probabilities derived from the language model to select the path maximizing joint likelihood. This approach transcends purely signal-level error correction by jointly optimizing channel observations and semantic coherence. Evaluated on convolutional codes with constraint length 3, the proposed scheme achieves approximately 1.5 dB gain in block error rate and improves semantic similarity by over 50% compared to traditional decoding methods.

Technology Category

• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Machine Learning: Information Theory

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Large language models for search
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

Traditional wireless communications rely solely on bit-level channel coding for error correction, without exploiting the inherent linguistic structure of the data source. This paper proposes a large language model (LLM) Viterbi decoder that integrates LLM priors into the Viterbi decoding for text transmission over AWGN channels. The proposed decoder maintains multiple candidate paths during the Viterbi decoding and periodically evaluates path reliabilities using a fine-tuned Byte-level T5 (ByT5) language model. By combining channel reliability metrics with semantic probability from the LLM, it outputs the path that maximizes the joint likelihood of channel observations and linguistic coherence. Simulations show that our decoder achieves significant performance gains over conventional Viterbi decoding in terms of both block error rate (BLER) and semantic similarity. For convolutional codes with constraint length 3, it achieves approximately 1.5 dB more coding gain in BLER, with over 50% improvements in semantic similarity. The framework can extend to other structured data sources beyond text.

Problem

• convolutional codes
• semantic-aware decoding
• LLM priors
• Viterbi decoding
• text transmission

Innovation

• LLM-Viterbi
• semantic-aware decoding
• convolutional codes
• ByT5 language model
• joint likelihood optimization