MiM-DiT: MoE in MoE with Diffusion Transformers for All-in-One Image Restoration

2026-03-03

AI Summary

Existing single models struggle to simultaneously address the diverse restoration demands posed by multiple image degradations—such as haze, blur, noise, and low-light conditions. To overcome this limitation, this work proposes a unified image restoration framework featuring an innovative “Mixture-of-Experts within Mixture-of-Experts” (MoE-in-MoE) dual-layer architecture. The outer Inter-MoE layer adaptively routes inputs to distinct expert groups based on degradation type, while the inner Intra-MoE layer further refines processing by handling sub-type variations within each degradation category, enabling coarse-to-fine adaptive modeling. Integrated into a diffusion Transformer, this approach synergistically combines pre-trained diffusion models with a dynamic expert selection mechanism. Extensive experiments demonstrate that the proposed method significantly outperforms current state-of-the-art techniques across diverse degradation scenarios, achieving both superior generalization and specialized restoration performance.

Technology Category

• Machine Learning: Mixture of Experts (MoE)
• Computer Vision: Diffusion Models for Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

All-in-one image restoration is challenging because different degradation types, such as haze, blur, noise, and low-light, impose diverse requirements on restoration strategies, making it difficult for a single model to handle them effectively. In this paper, we propose a unified image restoration framework that integrates a dual-level Mixture-of-Experts (MoE) architecture with a pretrained diffusion model. The framework operates at two levels: the Inter-MoE layer adaptively combines expert groups to handle major degradation types, while the Intra-MoE layer further selects specialized sub-experts to address fine-grained variations within each type. This design enables the model to achieve coarse-grained adaptation across diverse degradation categories while performing fine-grained modulation for specific intra-class variations, ensuring both high specialization in handling complex, real-world corruptions. Extensive experiments demonstrate that the proposed method performs favorably against the state-of-the-art approaches on multiple image restoration task.

Problem

• all-in-one image restoration
• image degradation
• haze
• blur
• noise

Innovation

• Mixture-of-Experts
• Diffusion Transformer
• All-in-One Image Restoration
• Dual-level MoE
• Pretrained Diffusion Model