MedGS: Gaussian Splatting for Multi-Modal 3D Medical Imaging

2025-09-20

AI Summary

To address low surface reconstruction and interpolation accuracy, along with prevalent artifacts in multimodal 3D medical imaging (e.g., ultrasound, MRI, CT), caused by noise corruption and inter-slice information sparsity, this paper proposes a semi-supervised neural implicit surface reconstruction framework based on Gaussian splatting. The method maps discrete 2D slices to a 3D Gaussian point cloud representation and integrates a lightweight neural implicit network to enable cross-modal, continuous geometric modeling and inter-slice interpolation. Compared to conventional approaches, our framework significantly improves noise robustness and structural fidelity, supports fine-grained editing, and enables accurate reconstruction of complex anatomical structures—while achieving higher training efficiency and reduced artifact generation. Experimental results demonstrate substantial improvements in reconstruction accuracy, computational efficiency, and clinical scalability across diverse modalities.

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Multi-modal three-dimensional (3D) medical imaging data, derived from ultrasound, magnetic resonance imaging (MRI), and potentially computed tomography (CT), provide a widely adopted approach for non-invasive anatomical visualization. Accurate modeling, registration, and visualization in this setting depend on surface reconstruction and frame-to-frame interpolation. Traditional methods often face limitations due to image noise and incomplete information between frames. To address these challenges, we present MedGS, a semi-supervised neural implicit surface reconstruction framework that employs a Gaussian Splatting (GS)-based interpolation mechanism. In this framework, medical imaging data are represented as consecutive two-dimensional (2D) frames embedded in 3D space and modeled using Gaussian-based distributions. This representation enables robust frame interpolation and high-fidelity surface reconstruction across imaging modalities. As a result, MedGS offers more efficient training than traditional neural implicit methods. Its explicit GS-based representation enhances noise robustness, allows flexible editing, and supports precise modeling of complex anatomical structures with fewer artifacts. These features make MedGS highly suitable for scalable and practical applications in medical imaging.

Problem

• Addressing limitations in 3D medical imaging from noise and incomplete data
• Improving frame interpolation and surface reconstruction across different imaging modalities
• Enhancing modeling accuracy and efficiency for complex anatomical structures

Innovation

• Gaussian Splatting-based interpolation for medical imaging
• Semi-supervised neural implicit surface reconstruction framework
• Explicit Gaussian representation for noise robustness