MoECodec: Image Compression for joint human and machine perception via Mixture-of-Experts

2026-06-18

AI Summary

Existing image compression methods struggle to simultaneously satisfy the requirements of human vision and diverse machine vision tasks within a single model, and they often lack dynamic adaptation to the semantic importance and complexity of different image regions. To address this, this work proposes MoECodec—a token-aware Mixture-of-Experts image compression framework that replaces conventional feed-forward network (FFN) layers in a Transformer architecture with a dynamic, token-level expert mixture mechanism. The framework employs a content- and task-aware routing strategy, stabilizes expert assignment via spatial total variation regularization, and introduces a lightweight Group Shuffle MLP as the expert structure to enable efficient and coherent computational resource allocation. Experiments demonstrate that MoECodec significantly outperforms existing approaches in both image reconstruction quality and performance across multiple downstream machine vision tasks, confirming the effectiveness and generalization capability of a unified multi-task compression model.

Abstract

Image compression for machines calls for a unified codec that serves multiple downstream vision tasks. Existing approaches either adopt task-specific end-to-end designs, raising parameter and deployment overhead, or rely on transfer-based adaptations that remain externally attached and heuristic task design. A key limitation shared by both lines of work is their largely static computation pattern, which applies similar transformations across tokens despite the fact that different image regions exhibit markedly different semantic importance and complexity for machine perception. We propose MoECodec, a token-aware image compression framework that supports multiple downstream tasks within a single model. MoECodec replaces the FFN layers in transformer-based compression model token-wise Mixture-of-Experts (MoE), enabling dynamic, token-level computation conditioned on the input content and task objective. To make MoE effective in compression model, we introduce a stable routing strategy that combines expert-choice routing with spatial total variation regularization to encourage spatially coherent assignments, and we propose a lightweight expert architecture, Group Shuffle MLP (GShMLP), to control parameter growth. Extensive experiments show consistent improvement against baselines on both conventional image reconstruction and machine tasks.

Problem

• image compression
• machine perception
• Mixture-of-Experts
• unified codec
• token-aware computation

Innovation

• Mixture-of-Experts
• token-aware compression
• dynamic computation
• image codec
• machine perception