BridgeSplat: Bidirectionally Coupled CT and Non-Rigid Gaussian Splatting for Deformable Intraoperative Surgical Navigation

2025-09-22

AI Summary

In deformable surgical navigation, bidirectional coupling between intraoperative monocular video and preoperative CT volumes remains challenging. Method: This paper proposes a joint optimization framework integrating 3D Gaussians with CT-derived triangular meshes. We design a parametric Gaussian representation coupled with mesh vertices and formulate a differentiable non-rigid deformation model under photometric supervision, enabling synchronized optimization of Gaussian parameters and mesh vertices; deformation gradients are backpropagated to dynamically update the preoperative CT volume. Contribution/Results: To our knowledge, this is the first work establishing a real-time, monocular RGB-video-driven deformable registration paradigm for CT volumes. Evaluated on porcine visceral surgery and synthetic human liver datasets, our method generates anatomically consistent and temporally coherent CT deformations using only monocular video input, effectively bridging the modality gap between intraoperative visual observations and preoperative volumetric data.

Technology Category

• Computer Vision: Learning & Optimization for CV
• Search and Optimization: Mixed Discrete/Continuous Search
• Constraint Satisfaction and Optimization: Mixed Discrete/Continuous Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments

Abstract

We introduce BridgeSplat, a novel approach for deformable surgical navigation that couples intraoperative 3D reconstruction with preoperative CT data to bridge the gap between surgical video and volumetric patient data. Our method rigs 3D Gaussians to a CT mesh, enabling joint optimization of Gaussian parameters and mesh deformation through photometric supervision. By parametrizing each Gaussian relative to its parent mesh triangle, we enforce alignment between Gaussians and mesh and obtain deformations that can be propagated back to update the CT. We demonstrate BridgeSplat's effectiveness on visceral pig surgeries and synthetic data of a human liver under simulation, showing sensible deformations of the preoperative CT on monocular RGB data. Code, data, and additional resources can be found at https://maxfehrentz.github.io/ct-informed-splatting/ .

Problem

• Bridges surgical video and preoperative CT data for navigation
• Couples 3D Gaussian splatting with CT mesh deformation optimization
• Enables monocular RGB-based deformation propagation to update CT

Innovation

• Bidirectional CT-Gaussian splatting coupling
• Joint Gaussian-mesh photometric optimization
• Mesh-relative Gaussian parametrization enabling deformation