Rapid patient-specific neural networks for intraoperative X-ray to volume registration

2025-03-20

AI Summary

To address poor generalizability, manual parameter tuning, and reliance on expert annotations in X-ray-guided 2D/3D registration, this work proposes a patient-specific neural registration framework. It leverages X-ray physics modeling and differentiable volume rendering to synthesize high-fidelity, annotation-free training data; integrates a lightweight network with end-to-end optimization to construct a single-patient model within five minutes. Evaluated on the largest multi-center clinical X-ray dataset to date, the method supports diverse anatomical sites and emergency scenarios, demonstrating robust cross-institutional and cross-modal performance. It achieves sub-millimeter accuracy (mean target registration error: 0.82 mm) and operates ten times faster than state-of-the-art methods, substantially lowering clinical deployment barriers.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Calibration & Uncertainty Quantification
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Systems and Infrastructure for Web, Mobile and WoT: Web applications in cross-disciplinary domains and verticals such as mixed reality, smart cities, and digital health

Abstract

The integration of artificial intelligence in image-guided interventions holds transformative potential, promising to extract 3D geometric and quantitative information from conventional 2D imaging modalities during complex procedures. Achieving this requires the rapid and precise alignment of 2D intraoperative images (e.g., X-ray) with 3D preoperative volumes (e.g., CT, MRI). However, current 2D/3D registration methods fail across the broad spectrum of procedures dependent on X-ray guidance: traditional optimization techniques require custom parameter tuning for each subject, whereas neural networks trained on small datasets do not generalize to new patients or require labor-intensive manual annotations, increasing clinical burden and precluding application to new anatomical targets. To address these challenges, we present xvr, a fully automated framework for training patient-specific neural networks for 2D/3D registration. xvr uses physics-based simulation to generate abundant high-quality training data from a patient's own preoperative volumetric imaging, thereby overcoming the inherently limited ability of supervised models to generalize to new patients and procedures. Furthermore, xvr requires only 5 minutes of training per patient, making it suitable for emergency interventions as well as planned procedures. We perform the largest evaluation of a 2D/3D registration algorithm on real X-ray data to date and find that xvr robustly generalizes across a diverse dataset comprising multiple anatomical structures, imaging modalities, and hospitals. Across surgical tasks, xvr achieves submillimeter-accurate registration at intraoperative speeds, improving upon existing methods by an order of magnitude. xvr is released as open-source software freely available at https://github.com/eigenvivek/xvr.

Problem

• Rapid alignment of 2D intraoperative X-ray with 3D preoperative volumes
• Overcoming generalization issues of neural networks in new patients
• Reducing clinical burden with automated, patient-specific training framework

Innovation

• Automated patient-specific neural network training
• Physics-based simulation for training data generation
• Submillimeter-accurate intraoperative registration speed