Learned Image Transmission with Hierarchical Variational Autoencoder

2024-08-29

arXiv.org

AI Summary

To jointly optimize efficiency and robustness for multi-resolution image transmission over noisy channels, this paper proposes an end-to-end joint source-channel coding (JSCC) framework based on a hierarchical variational autoencoder (HVAE). The method employs bidirectional autoregressive modeling to generate multi-scale latent representations and directly maps them to channel symbols; it further incorporates probabilistic noise channel modeling and an explicit feedback mechanism to enable dynamic bandwidth adaptation and feedback-driven generative decoding. Unlike existing JSCC approaches, our framework is the first to unify hierarchical architecture, autoregressive priors, and feedback-enhanced decoding. It achieves significant improvements in both rate-distortion performance and robustness to channel noise, establishing new state-of-the-art results on multi-resolution image transmission tasks.

Technology Category

• Machine Learning: Deep Generative Models & Autoencoders
• Computer Vision: Multi-modal Vision
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

In this paper, we introduce an innovative hierarchical joint source-channel coding (HJSCC) framework for image transmission, utilizing a hierarchical variational autoencoder (VAE). Our approach leverages a combination of bottom-up and top-down paths at the transmitter to autoregressively generate multiple hierarchical representations of the original image. These representations are then directly mapped to channel symbols for transmission by the JSCC encoder. We extend this framework to scenarios with a feedback link, modeling transmission over a noisy channel as a probabilistic sampling process and deriving a novel generative formulation for JSCC with feedback. Compared with existing approaches, our proposed HJSCC provides enhanced adaptability by dynamically adjusting transmission bandwidth, encoding these representations into varying amounts of channel symbols. Extensive experiments on images of varying resolutions demonstrate that our proposed model outperforms existing baselines in rate-distortion performance and maintains robustness against channel noise. The source code will be made available upon acceptance.

Problem

• Hierarchical image transmission optimization
• Adaptive bandwidth adjustment for transmission
• Enhanced robustness to channel noise

Innovation

• Hierarchical VAE for image transmission
• Dynamic bandwidth adjustment in HJSCC
• Feedback link enhances transmission robustness