Improving 2D Diffusion Models for 3D Medical Imaging with Inter-Slice Consistent Stochasticity

2026-02-04

AI Summary

This work addresses the issue of inter-slice discontinuity commonly observed in 3D medical image reconstruction using 2D diffusion models, which arises from the randomness inherent in slice-wise sampling. To mitigate this, the authors propose a plug-and-play Inter-Slice Consistent Sampling (ISCS) strategy that aligns the random noise across adjacent slices during the diffusion sampling process. This approach enhances inter-slice consistency without introducing additional loss terms, hyperparameters, or computational overhead. ISCS is compatible with various 3D reconstruction pipelines that leverage 2D pre-trained diffusion models and demonstrates consistent improvements in both inter-slice continuity and overall reconstruction fidelity across multiple medical imaging tasks. The results validate that explicitly controlling cross-slice randomness is an effective means to improve the quality of 3D reconstructions derived from 2D diffusion models.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

3D medical imaging is in high demand and essential for clinical diagnosis and scientific research. Currently, diffusion models (DMs) have become an effective tool for medical imaging reconstruction thanks to their ability to learn rich, high-quality data priors. However, learning the 3D data distribution with DMs in medical imaging is challenging, not only due to the difficulties in data collection but also because of the significant computational burden during model training. A common compromise is to train the DMs on 2D data priors and reconstruct stacked 2D slices to address 3D medical inverse problems. However, the intrinsic randomness of diffusion sampling causes severe inter-slice discontinuities of reconstructed 3D volumes. Existing methods often enforce continuity regularizations along the z-axis, which introduces sensitive hyper-parameters and may lead to over-smoothing results. In this work, we revisit the origin of stochasticity in diffusion sampling and introduce Inter-Slice Consistent Stochasticity (ISCS), a simple yet effective strategy that encourages interslice consistency during diffusion sampling. Our key idea is to control the consistency of stochastic noise components during diffusion sampling, thereby aligning their sampling trajectories without adding any new loss terms or optimization steps. Importantly, the proposed ISCS is plug-and-play and can be dropped into any 2D trained diffusion based 3D reconstruction pipeline without additional computational cost. Experiments on several medical imaging problems show that our method can effectively improve the performance of medical 3D imaging problems based on 2D diffusion models. Our findings suggest that controlling inter-slice stochasticity is a principled and practically attractive route toward high-fidelity 3D medical imaging with 2D diffusion priors. The code is available at: https://github.com/duchenhe/ISCS

Problem

• 3D medical imaging
• diffusion models
• inter-slice consistency
• stochasticity
• image reconstruction

Innovation

• Inter-Slice Consistent Stochasticity
• diffusion models
• 3D medical imaging
• stochasticity control
• plug-and-play reconstruction