Investigating the Feasibility of Patch-based Inference for Generalized Diffusion Priors in Inverse Problems for Medical Images

2025-01-25

AI Summary

To address the high computational cost and excessive GPU memory consumption of full-image diffusion models in MRI inverse problem solving, this work systematically validates and implements patch-based diffusion prior modeling for the first time. We propose a lightweight adaptation mechanism and an overlapping patch fusion strategy to effectively suppress blocking artifacts. The diffusion prior is seamlessly integrated into the Plug-and-Play (PnP) optimization framework, enabling joint evaluation across multiple tasks (e.g., super-resolution, reconstruction) and diverse datasets. Experiments demonstrate that our approach achieves reconstruction quality comparable to full-image inference while reducing GPU memory usage by 62% and accelerating inference by 2.3×. Moreover, it exhibits cross-PnP algorithm compatibility—i.e., plug-and-play capability without retraining. This work establishes a new paradigm for efficient and scalable generative prior modeling in medical imaging.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Optimization
• Search and Optimization: Non-convex Optimization

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Web Mining and Content Analysis: Content-based information diffusion
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Plug-and-play approaches to solving inverse problems such as restoration and super-resolution have recently benefited from Diffusion-based generative priors for natural as well as medical images. However, solutions often use the standard albeit computationally intensive route of training and inferring with the whole image on the diffusion prior. While patch-based approaches to evaluating diffusion priors in plug-and-play methods have received some interest, they remain an open area of study. In this work, we explore the feasibility of the usage of patches for training and inference of a diffusion prior on MRI images. We explore the minor adaptation necessary for artifact avoidance, the performance and the efficiency of memory usage of patch-based methods as well as the adaptability of whole image training to patch-based evaluation - evaluating across multiple plug-and-play methods, tasks and datasets.

Problem

• Medical Image Processing
• MRI Images
• Patch-based Inference

Innovation

• Generalized Diffusion Prior
• Patch-based Inference
• Efficient MRI Image Processing