DiT-JSCC: Rethinking Deep JSCC with Diffusion Transformers and Semantic Representations

2026-01-06

AI Summary

This work addresses the challenge that existing deep joint source-channel coding (Deep JSCC) methods struggle to simultaneously preserve semantic consistency and visual fidelity under extreme wireless channel conditions, primarily due to the encoder’s lack of explicit semantic discrimination capability. To overcome this limitation, we propose DiT-JSCC, which for the first time aligns semantic representations with a diffusion Transformer (DiT) decoder. Our approach introduces a semantics-prioritized dual-branch encoder and a coarse-to-fine conditional generative decoder within a collaborative learning framework. Furthermore, we incorporate a training-free adaptive bandwidth allocation mechanism grounded in Kolmogorov complexity. Extensive experiments demonstrate that DiT-JSCC significantly outperforms current JSCC schemes under ultra-low bandwidth and low signal-to-noise ratio conditions, achieving state-of-the-art performance in both semantic accuracy and image quality.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Search and Optimization: Learning to Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Generative joint source-channel coding (GJSCC) has emerged as a new Deep JSCC paradigm for achieving high-fidelity and robust image transmission under extreme wireless channel conditions, such as ultra-low bandwidth and low signal-to-noise ratio. Recent studies commonly adopt diffusion models as generative decoders, but they frequently produce visually realistic results with limited semantic consistency. This limitation stems from a fundamental mismatch between reconstruction-oriented JSCC encoders and generative decoders, as the former lack explicit semantic discriminability and fail to provide reliable conditional cues. In this paper, we propose DiT-JSCC, a novel GJSCC backbone that can jointly learn a semantics-prioritized representation encoder and a diffusion transformer (DiT) based generative decoder, our open-source project aims to promote the future research in GJSCC. Specifically, we design a semantics-detail dual-branch encoder that aligns naturally with a coarse-to-fine conditional DiT decoder, prioritizing semantic consistency under extreme channel conditions. Moreover, a training-free adaptive bandwidth allocation strategy inspired by Kolmogorov complexity is introduced to further improve the transmission efficiency, thereby indeed redefining the notion of information value in the era of generative decoding. Extensive experiments demonstrate that DiT-JSCC consistently outperforms existing JSCC methods in both semantic consistency and visual quality, particularly in extreme regimes.

Problem

• Generative JSCC
• Semantic Consistency
• Diffusion Models
• Wireless Image Transmission
• Joint Source-Channel Coding

Innovation

• Diffusion Transformer
• Semantic Representation
• Joint Source-Channel Coding
• Generative JSCC
• Adaptive Bandwidth Allocation