All-in-One Medical Image Restoration with Latent Diffusion-Enhanced Vector-Quantized Codebook Prior

2025-07-26

AI Summary

This paper addresses the prior conflict problem arising from heterogeneous degradations (e.g., MRI super-resolution, CT denoising, PET synthesis) in multi-task medical image restoration. We propose DiffCode, a unified framework that constructs a task-adaptive vector-quantized codebook library to explicitly encode task-specific priors, and integrates latent diffusion strategies to iteratively refine latent-space feature distributions—enabling joint optimization of codebook retrieval and image reconstruction. DiffCode supports end-to-end, multi-modal, multi-degradation restoration within a single architecture, eliminating the need for task-specific model design. Experiments demonstrate that DiffCode consistently outperforms state-of-the-art methods across multiple medical image restoration benchmarks, achieving average improvements of 1.2–2.8 dB in PSNR and 0.015–0.032 in SSIM. Critically, reconstructed images preserve anatomical fidelity and maintain clinical interpretability.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

All-in-one medical image restoration (MedIR) aims to address multiple MedIR tasks using a unified model, concurrently recovering various high-quality (HQ) medical images (e.g., MRI, CT, and PET) from low-quality (LQ) counterparts. However, all-in-one MedIR presents significant challenges due to the heterogeneity across different tasks. Each task involves distinct degradations, leading to diverse information losses in LQ images. Existing methods struggle to handle these diverse information losses associated with different tasks. To address these challenges, we propose a latent diffusion-enhanced vector-quantized codebook prior and develop extbf{DiffCode}, a novel framework leveraging this prior for all-in-one MedIR. Specifically, to compensate for diverse information losses associated with different tasks, DiffCode constructs a task-adaptive codebook bank to integrate task-specific HQ prior features across tasks, capturing a comprehensive prior. Furthermore, to enhance prior retrieval from the codebook bank, DiffCode introduces a latent diffusion strategy that utilizes the diffusion model's powerful mapping capabilities to iteratively refine the latent feature distribution, estimating more accurate HQ prior features during restoration. With the help of the task-adaptive codebook bank and latent diffusion strategy, DiffCode achieves superior performance in both quantitative metrics and visual quality across three MedIR tasks: MRI super-resolution, CT denoising, and PET synthesis.

Problem

• Address multiple medical image restoration tasks with a unified model
• Compensate for diverse information losses across different imaging tasks
• Enhance prior retrieval using latent diffusion strategy for accurate restoration

Innovation

• Latent diffusion-enhanced vector-quantized codebook prior
• Task-adaptive codebook bank for diverse information losses
• Iterative refinement of latent feature distribution