Mixture-of-Experts-based Entropy Model for Learned Image Compression

2026-08-11

AI Summary

This work addresses the limited efficiency of learning-based image compression by introducing, for the first time, a Mixture-of-Experts (MoE) mechanism into entropy model design. The proposed approach employs a conditionally sparse activation architecture based on MoE, which dynamically selects expert subnetworks to enable adaptive, end-to-end image compression. This strategy significantly reduces computational overhead while preserving reconstruction quality. Evaluated on the Kodak dataset, the method achieves a BD-Rate gain of −16.85% compared to the latest video coding standard, VVC, demonstrating its superior compression efficiency and effective exploitation of model sparsity.

Abstract

Learned image compression has seen significant progress in recent years with the development of end-to-end learned models that achieve better compression efficiency than state-of-the-art conventional methods. Recently, Mixture of Experts (MoE) approaches have seen promising results in NLP and computer vision tasks. In this paper, we introduce the MoE approach to learned image compression. We propose a MoE-based Entropy model (MoEE) for learned image compression, allowing the model to selectively activate only the subset of parameters required for the input image. Our model achieves a BD-Rate improvement over VVC of -16.85% on the Kodak dataset.

Problem

• Learned Image Compression
• Entropy Model
• Mixture of Experts
• Compression Efficiency
• BD-Rate

Innovation

• Mixture of Experts
• Learned Image Compression
• Entropy Model
• BD-Rate
• Neural Compression