DiffUS: Differentiable Ultrasound Rendering from Volumetric Imaging

2025-08-08

AI Summary

Intraoperative ultrasound (iUS) images suffer from severe noise, prominent artifacts, and poor registration with preoperative MRI/CT, leading to a critical disconnect between preoperative planning and intraoperative guidance. Method: We propose the first fully differentiable, physics-based ultrasound rendering framework implemented end-to-end in PyTorch. It jointly models acoustic impedance mapping, multi-path reflection, ray-traced wave propagation under扇-shaped geometry, sparse linear system solving, and deep analytical echo synthesis. Unlike conventional non-differentiable simulators or purely data-driven approaches, our framework enables gradient backpropagation and can be directly embedded into registration and reconstruction optimization pipelines. Results: Evaluated on the ReMIND brain dataset, our method generates anatomically accurate iUS images that faithfully reproduce key degradations—including speckle noise and depth-dependent attenuation—yielding significant improvements in cross-modal alignment accuracy and intraoperative guidance reliability.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Planning, Routing, and Scheduling: Replanning and Plan Repair

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Intraoperative ultrasound imaging provides real-time guidance during numerous surgical procedures, but its interpretation is complicated by noise, artifacts, and poor alignment with high-resolution preoperative MRI/CT scans. To bridge the gap between reoperative planning and intraoperative guidance, we present DiffUS, a physics-based, differentiable ultrasound renderer that synthesizes realistic B-mode images from volumetric imaging. DiffUS first converts MRI 3D scans into acoustic impedance volumes using a machine learning approach. Next, we simulate ultrasound beam propagation using ray tracing with coupled reflection-transmission equations. DiffUS formulates wave propagation as a sparse linear system that captures multiple internal reflections. Finally, we reconstruct B-mode images via depth-resolved echo extraction across fan-shaped acquisition geometry, incorporating realistic artifacts including speckle noise and depth-dependent degradation. DiffUS is entirely implemented as differentiable tensor operations in PyTorch, enabling gradient-based optimization for downstream applications such as slice-to-volume registration and volumetric reconstruction. Evaluation on the ReMIND dataset demonstrates DiffUS's ability to generate anatomically accurate ultrasound images from brain MRI data.

Problem

• Bridging preoperative MRI/CT with intraoperative ultrasound guidance
• Synthesizing realistic B-mode images from volumetric scans
• Enabling gradient-based optimization for surgical applications

Innovation

• Machine learning converts MRI to acoustic impedance
• Ray tracing simulates ultrasound beam propagation
• Differentiable tensor operations enable gradient optimization