Multiple Scale Latents for Learned Image Compression

2026-08-11

AI Summary

This work addresses the limitation of existing learned image compression methods that rely on a single latent representation, which struggles to effectively capture multi-scale spatial structures. To overcome this, the authors propose a hierarchical multi-scale latent representation mechanism, wherein each scale-specific latent variable is paired with an independent entropy model. This framework is integrated with hyperprior techniques and trained end-to-end, substantially enhancing the expressiveness of the entropy model. The proposed approach is orthogonal to mainstream learned compression architectures, offering strong generality and seamless integrability. Experimental results on the Kodak dataset demonstrate a 17.9% BD-rate gain over VVC, confirming its superior rate-distortion performance.

Abstract

Most learned image compression systems rely on a single latent representation combined with a hyperprior, which limits their ability to efficiently capture image structure across spatial scales. In this work, we propose a hierarchical latent representation to improve the efficiency of the entropy model. By using multiple latents at different scales, each with its own entropy model, we better capture the spatial structure of the latent representation. Our experiments show that this approach achieves a 17.9% BD-rate reduction over VVC on Kodak, demonstrating the effectiveness of multi-scale latent representations. Furthermore, the approach is orthogonal to other advances in learned image compression, making it a versatile addition to existing methods.

Problem

• learned image compression
• latent representation
• multi-scale
• spatial structure
• entropy model

Innovation

• multi-scale latents
• learned image compression
• hierarchical latent representation
• entropy modeling
• BD-rate reduction