Distributed Image Compression with Multimodal Side Information at Extremely Low Bitrates

2026-05-21

AI Summary

This work addresses the challenge of severe detail loss and blurry reconstructions in existing distributed image compression methods at extremely low bitrates (<0.1 bpp), where side information is underutilized. To overcome this limitation, we propose a Multimodal Distributed Image Compression (MDIC) framework that uniquely leverages both textual and visual side information. Our approach introduces a text-guided diffusion decoder and a feature mask generator supervised by fine-grained alignment, jointly recovering semantically consistent global structures and local details. By integrating text-to-image diffusion models, VQ-VAE quantized embeddings, and a feature mask guidance mechanism, MDIC achieves state-of-the-art perceptual quality, significantly outperforming current methods on the KITTI Stereo and Cityscapes datasets.

Abstract

Distributed Image Compression (DIC) is crucial for multi-view transmission, especially when operating at extremely low bitrates (< 0.1 bpp). Its core challenge is effectively utilizing side information to achieve high-quality reconstruction under strict bitrate budgets. However, existing DIC approaches struggle to exploit global context and object-level details from side information, leading to local blurring and the loss of fine details in the reconstruction. To address these limitations, we propose a Multimodal DIC framework (MDIC), which, for the first time, leverages side information in a multimodal manner into the DIC paradigm, effectively preserving fine-grained local details and enhancing global perceptual quality in reconstructed images. Specifically, we introduce a text-to-image diffusion-based decoder conditioned on textual side information extracted from correlated images to capture shared global semantics. Moreover, we design a feature-mask generator, supervised by a multimodal fine-grained alignment task, to strengthen the exploitation of visual side information. The generated mask serves two purposes: first, it guides the extraction of fine-grained details from losslessly transmitted side information to preserve the semantic consistency of reconstructed details; second, it regulates the extraction of clustered feature representations from the quantized VQ-VAE embeddings, compensating for category information lost under the extreme compression of the primary image. Extensive experiments on the widely used KITTI Stereo and Cityscapes datasets demonstrate that MDIC achieves state-of-the-art perceptual quality at extremely low bitrates.

Problem

• Distributed Image Compression
• Extremely Low Bitrates
• Side Information
• Image Reconstruction
• Multimodal

Innovation

• Multimodal Side Information
• Distributed Image Compression
• Diffusion-based Decoder
• Feature-Mask Generator
• Extremely Low Bitrates