Separate Source Channel Coding Is Still What You Need: An LLM-based Rethinking

2025-01-08

AI Summary

This study challenges the prevailing joint source-channel coding (JSCC) paradigm in semantic communication, identifying fundamental bottlenecks—including excessive floating-point computational overhead and poor compatibility with legacy digital systems—under high-fidelity digital transmission. To address these limitations, we propose a novel separated source-channel coding (SSCC) architecture. Specifically, we pioneer the use of large language models (LLMs) for semantic source encoding and integrate them with an error-correcting code Transformer (ECCT) for efficient channel decoding. Experimental results demonstrate that the proposed SSCC framework significantly outperforms JSCC in both semantic fidelity and bit-error rate, while preserving full compatibility with conventional digital infrastructure. Moreover, SSCC enhances optimization flexibility and deployment adaptability. Our findings substantiate that SSCC remains not only highly competitive in performance but also practically viable for modern semantic communication systems.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Search and Optimization: Mixed Discrete/Continuous Search
• Multiagent Systems: Agent Communication

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

Along with the proliferating research interest in Semantic Communication (SemCom), Joint Source Channel Coding (JSCC) has dominated the attention due to the widely assumed existence in efficiently delivering information semantics. %has emerged as a pivotal area of research, aiming to enhance the efficiency and reliability of information transmission through deep learning-based methods. Nevertheless, this paper challenges the conventional JSCC paradigm, and advocates for adoption of Separate Source Channel Coding (SSCC) to enjoy the underlying more degree of freedom for optimization. We demonstrate that SSCC, after leveraging the strengths of Large Language Model (LLM) for source coding and Error Correction Code Transformer (ECCT) complemented for channel decoding, offers superior performance over JSCC. Our proposed framework also effectively highlights the compatibility challenges between SemCom approaches and digital communication systems, particularly concerning the resource costs associated with the transmission of high precision floating point numbers. Through comprehensive evaluations, we establish that empowered by LLM-based compression and ECCT-enhanced error correction, SSCC remains a viable and effective solution for modern communication systems. In other words, separate source and channel coding is still what we need!

Problem

• Semantic Communication Systems
• Source Channel Coding
• Efficiency and Compatibility

Innovation

• Independent Source Channel Coding
• Error Correction Codes
• Semantic Communication Compatibility