DeepStream: Prototyping Deep Joint Source-Channel Coding for Real-Time Multimedia Transmissions

2025-09-07

AI Summary

To address insufficient multimedia transmission efficiency and reliability under low-SNR conditions in 6G systems, this paper proposes an end-to-end deep joint source-channel coding (DeepJSCC) framework. Methodologically, it builds upon an OFDM architecture integrated with deep neural networks, introducing two key innovations: (i) a feature-to-symbol mapping mechanism and (ii) cross-subcarrier precoding to enhance subcarrier independence and reduce peak-to-average power ratio (PAPR); additionally, a progressive encoding strategy is designed to dynamically adapt to QoS requirements while respecting latency constraints. Contributions include the first real-time prototype validation of image/video streaming on a software-defined radio (SDR) platform. At 10 dB SNR, the framework achieves 35 dB PSNR for images and 20 dB MS-SSIM for video—significantly outperforming conventional separate source-channel coding schemes, which fail to reconstruct intelligible content. This work establishes a deployable, end-to-end learning paradigm for intelligent air-interface coding in 6G.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Computer Vision: Multi-modal Vision
• Machine Learning: Deep Generative Models & Autoencoders

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Data management and stream processing for Web, mobile and wireless applications
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Deep learning-based joint source-channel coding (DeepJSCC) has emerged as a promising technique in 6G for enhancing the efficiency and reliability of data transmission across diverse modalities, particularly in low signal-to-noise ratio (SNR) environments. This advantage is realized by leveraging powerful neural networks to learn an optimal end-to-end mapping from the source data directly to the transmit symbol sequence, eliminating the need for separate source coding, channel coding, and modulation. Although numerous efforts have been made towards efficient DeepJSCC, they have largely stayed at numerical simulations that can be far from practice, leaving the real-world viability of DeepJSCC largely unverified. To this end, we prototype DeepStream upon orthogonal frequency division multiplexing (OFDM) technology to offer efficient and robust DeepJSCC for multimedia transmission. In conforming to OFDM, we develop both a feature-to-symbol mapping method and a cross-subcarrier precoding method to improve the subcarrier independence and reduce peak-to-average power ratio. To reduce system complexity and enable flexibility in accommodating varying quality of service requirements, we further propose a progressive coding strategy that adjusts the compression ratio based on latency with minimal performance loss. We implement DeepStream for real-time image transmission and video streaming using software-defined radio. Extensive evaluations verify that DeepStream outperforms both the standard scheme and the direct deployment scheme. Particularly, at an SNR of 10 dB, DeepStream achieves a PSNR of 35 dB for image transmission and an MS-SSIM of 20 dB for video streaming, whereas the standard scheme fails to recover meaningful information.

Problem

• Developing DeepJSCC for real-time multimedia transmission
• Prototyping DeepJSCC on OFDM to improve efficiency and robustness
• Enhancing subcarrier independence and reducing power ratio in OFDM

Innovation

• Prototyping DeepJSCC on OFDM for multimedia transmission
• Developing feature-symbol mapping and cross-subcarrier precoding methods
• Proposing progressive coding strategy for adaptive compression