Domain-Specific Latent Representations Improve the Fidelity of Diffusion-Based Medical Image Super-Resolution

2026-04-13

AI Summary

This work addresses the limitations of general-purpose variational autoencoders (VAEs), such as those in Stable Diffusion, which constrain reconstruction fidelity in medical image super-resolution. The study demonstrates for the first time that the VAE—not the diffusion architecture—is the primary performance bottleneck. To overcome this, the authors propose MedVAE, a VAE specifically designed for medical imaging, which significantly enhances reconstruction quality while keeping the rest of the diffusion model unchanged. MedVAE is pretrained on 1.6 million medical images and systematically evaluated through ablation studies incorporating latent diffusion models, wavelet analysis, and multiple scheduling strategies. Experiments on knee MRI, brain MRI, and chest X-ray datasets show PSNR improvements of 2.91–3.29 dB without increasing hallucination rates. Furthermore, VAE reconstruction quality strongly predicts downstream super-resolution performance (R² = 0.67).

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Latent diffusion models for medical image super-resolution universally inherit variational autoencoders designed for natural photographs. We show that this default choice, not the diffusion architecture, is the dominant constraint on reconstruction quality. In a controlled experiment holding all other pipeline components fixed, replacing the generic Stable Diffusion VAE with MedVAE, a domain-specific autoencoder pretrained on more than 1.6 million medical images, yields +2.91 to +3.29 dB PSNR improvement across knee MRI, brain MRI, and chest X-ray (n = 1,820; Cohen's d = 1.37 to 1.86, all p < 10^{-20}, Wilcoxon signed-rank). Wavelet decomposition localises the advantage to the finest spatial frequency bands encoding anatomically relevant fine structure. Ablations across inference schedules, prediction targets, and generative architectures confirm the gap is stable within plus or minus 0.15 dB, while hallucination rates remain comparable between methods (Cohen's h < 0.02 across all datasets), establishing that reconstruction fidelity and generative hallucination are governed by independent pipeline components. These results provide a practical screening criterion: autoencoder reconstruction quality, measurable without diffusion training, predicts downstream SR performance (R^2 = 0.67), suggesting that domain-specific VAE selection should precede diffusion architecture search. Code and trained model weights are publicly available at https://github.com/sebasmos/latent-sr.

Problem

• medical image super-resolution
• latent diffusion models
• domain-specific representations
• variational autoencoders
• reconstruction fidelity

Innovation

• domain-specific VAE
• medical image super-resolution
• latent diffusion models
• reconstruction fidelity
• generative hallucination