Robust and Accurate Multi-view 2D/3D Image Registration with Differentiable X-ray Rendering and Dual Cross-view Constraints

2025-06-27

AI Summary

To address the limited intraoperative field-of-view in single-view X-ray imaging—which undermines the robustness of preoperative 3D model-to-intraoperative 2D image registration—this paper proposes a multi-view 2D/3D medical image registration method. Our approach introduces three key contributions: (1) a differentiable X-ray renderer enabling end-to-end gradient propagation; (2) dual cross-view constraints integrating pose consistency and normalized cross-correlation-based image similarity; and (3) a two-stage optimization strategy—coarse registration followed by test-time refinement leveraging geometric constraints across multi-view projections. Evaluated on six cadaveric specimens from the DeepFluoro dataset, our method achieves a mean target registration error of 0.79 ± 2.17 mm, significantly outperforming current state-of-the-art methods.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multi-instance/Multi-view Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Systems and Infrastructure for Web, Mobile and WoT: Web applications in cross-disciplinary domains and verticals such as mixed reality, smart cities, and digital health

Abstract

Robust and accurate 2D/3D registration, which aligns preoperative models with intraoperative images of the same anatomy, is crucial for successful interventional navigation. To mitigate the challenge of a limited field of view in single-image intraoperative scenarios, multi-view 2D/3D registration is required by leveraging multiple intraoperative images. In this paper, we propose a novel multi-view 2D/3D rigid registration approach comprising two stages. In the first stage, a combined loss function is designed, incorporating both the differences between predicted and ground-truth poses and the dissimilarities (e.g., normalized cross-correlation) between simulated and observed intraoperative images. More importantly, additional cross-view training loss terms are introduced for both pose and image losses to explicitly enforce cross-view constraints. In the second stage, test-time optimization is performed to refine the estimated poses from the coarse stage. Our method exploits the mutual constraints of multi-view projection poses to enhance the robustness of the registration process. The proposed framework achieves a mean target registration error (mTRE) of $0.79 pm 2.17$ mm on six specimens from the DeepFluoro dataset, demonstrating superior performance compared to state-of-the-art registration algorithms.

Problem

• Aligns preoperative models with intraoperative images accurately
• Mitigates limited field of view in single-image scenarios
• Enhances robustness using multi-view projection constraints

Innovation

• Differentiable X-ray rendering for image simulation
• Dual cross-view constraints for robust registration
• Two-stage coarse-to-fine pose optimization