On the Rate-Distortion-Complexity Tradeoff for Semantic Communication

2026-02-16

AI Summary

This work addresses the neglect of high computational complexity in existing deep learning–based semantic communication systems and the lack of a theoretical characterization of the trade-off among transmission rate, semantic fidelity, and model complexity. To this end, the authors propose a Rate–Distortion–Complexity (RDC) theoretical framework that extends classical rate–distortion theory to semantic communication scenarios. The framework introduces a semantic distance combining bit-wise distortion and statistical divergence, and defines model complexity via the Minimum Description Length and Information Bottleneck principles. Closed-form achievable rates are derived for Gaussian and binary semantic sources, revealing the underlying triple trade-off. Empirical validation on real image and video data demonstrates the efficacy of the theoretical analysis, with the proposed complexity metric showing strong correlation with actual computational overhead, thereby enabling efficient semantic communication system design under resource constraints.

Technology Category

• Machine Learning: Information Theory
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Semantic communication is a novel communication paradigm that focuses on conveying the user's intended meaning rather than the bit-wise transmission of source signals. One of the key challenges is to effectively represent and extract the semantic meaning of any given source signals. While deep learning (DL)-based solutions have shown promising results in extracting implicit semantic information from a wide range of sources, existing work often overlooks the high computational complexity inherent in both model training and inference for the DL-based encoder and decoder. To bridge this gap, this paper proposes a rate-distortion-complexity (RDC) framework which extends the classical rate-distortion theory by incorporating the constraints on semantic distance, including both the traditional bit-wise distortion metric and statistical difference-based divergence metric, and complexity measure, adopted from the theory of minimum description length and information bottleneck. We derive the closed-form theoretical results of the minimum achievable rate under given constraints on semantic distance and complexity for both Gaussian and binary semantic sources. Our theoretical results show a fundamental three-way tradeoff among achievable rate, semantic distance, and model complexity. Extensive experiments on real-world image and video datasets validate this tradeoff and further demonstrate that our information-theoretic complexity measure effectively correlates with practical computational costs, guiding efficient system design in resource-constrained scenarios.

Problem

• semantic communication
• rate-distortion-complexity tradeoff
• computational complexity
• semantic distortion
• information bottleneck

Innovation

• Semantic Communication
• Rate-Distortion-Complexity Tradeoff
• Information Bottleneck
• Minimum Description Length
• Computational Complexity