NAVIUS: Navigated Augmented Reality Visualization for Ureteroscopic Surgery

2025-03-21

AI Summary

During flexible ureteroscopy, the complex anatomy of the renal collecting system and limited endoscopic field-of-view contribute to high stone miss rates among novices, necessitating secondary procedures in approximately one-third of patients within 20 months. To address this, we propose NAVIUS: the first lightweight augmented reality (AR) navigation framework specifically designed for urological endoscopy. NAVIUS integrates real-time electromagnetic tracking with AR visualization via HoloLens 2, overlaying preoperative imaging, patient-specific 3D anatomical models, and endoscope pose estimates onto the surgical view. Leveraging OpenIGTLink, it enables low-latency, cross-platform coordination between electromagnetic tracking hardware, 3D Slicer, and Unity. Clinical validity was first demonstrated in a kidney phantom study. Results show that NAVIUS increased surgeons’ coverage of the renal collecting system by 23.73% on average and reduced subjective workload (NASA-TLX) by up to 27.27%. All source code is publicly available.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Multi-modal Vision
• Planning, Routing, and Scheduling: Activity and Plan Recognition

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Security and Privacy: Data transparency and provenance

Abstract

Ureteroscopy is the standard of care for diagnosing and treating kidney stones and tumors. However, current ureteroscopes have a limited field of view, requiring significant experience to adequately navigate the renal collecting system. This is evidenced by the fact that inexperienced surgeons have higher rates of missed stones. One-third of patients with residual stones require re-operation within 20 months. In order to aid surgeons to fully explore the kidney, this study presents the Navigated Augmented Reality Visualization for Ureteroscopic Surgery (NAVIUS) system. NAVIUS assists surgeons by providing 3D maps of the target anatomy, real-time scope positions, and preoperative imaging overlays. To enable real-time navigation and visualization, we integrate an electromagnetic tracker-based navigation pipeline with augmented reality visualizations. NAVIUS connects to 3D Slicer and Unity with OpenIGTLink, and uses HoloLens 2 as a holographic interface. We evaluate NAVIUS through a user study where surgeons conducted ureteroscopy on kidney phantoms with and without visual guidance. With our proposed system, we observed that surgeons explored more areas within the collecting system with NAVIUS (average 23.73% increase), and NASA-TLX metrics were improved (up to 27.27%). NAVIUS acts as a step towards better surgical outcomes and surgeons' experience. The codebase for the system will be available at: https://github.com/vu-maple-lab/NAVIUS.

Problem

• Enhances ureteroscopic surgery navigation with 3D maps
• Reduces missed stones by improving field of view
• Integrates augmented reality for real-time surgical guidance

Innovation

• 3D maps and real-time scope positions
• Electromagnetic tracker-based navigation pipeline
• Augmented reality via HoloLens 2 interface