Markerless Augmented Reality Registration for Surgical Guidance: A Multi-Anatomy Clinical Accuracy Study

2025-11-03

AI Summary

Achieving high-precision, fiducial-free augmented reality (AR) registration for small or low-curvature anatomical regions—such as the foot, ear, and calf—remains challenging in real surgical settings. Method: This study proposes a purely depth-driven, fiducial-free AR registration framework integrating human-in-the-loop initialization with a global–local multi-stage registration strategy. Leveraging HoloLens 2’s AHAT depth sensor, depth bias correction, and AR tracking capabilities, it enables intraoperative, markerless registration of small anatomical structures for the first time. Contribution/Results: Clinical validation yields a median target registration error (TRE) of 3.9 mm overall—3.2 mm (foot), 4.3 mm (ear), and 5.3 mm (calf)—with >90% of localization errors below 5 mm, approaching the clinical accuracy threshold for moderate-risk procedures. This work establishes a clinically deployable paradigm for fiducial-free AR surgical navigation.

Technology Category

• Intelligent Robots: Localization, Mapping, and Navigation
• Computer Vision: Motion & Tracking
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Purpose: In this paper, we develop and clinically evaluate a depth-only, markerless augmented reality (AR) registration pipeline on a head-mounted display, and assess accuracy across small or low-curvature anatomies in real-life operative settings. Methods: On HoloLens 2, we align Articulated HAnd Tracking (AHAT) depth to Computed Tomography (CT)-derived skin meshes via (i) depth-bias correction, (ii) brief human-in-the-loop initialization, (iii) global and local registration. We validated the surface-tracing error metric by comparing"skin-to-bone"relative distances to CT ground truth on leg and foot models, using an AR-tracked tool. We then performed seven intraoperative target trials (feet x2, ear x3, leg x2) during the initial stage of fibula free-flap harvest and mandibular reconstruction surgery, and collected 500+ data per trial. Results: Preclinical validation showed tight agreement between AR-traced and CT distances (leg: median |Delta d| 0.78 mm, RMSE 0.97 mm; feet: 0.80 mm, 1.20 mm). Clinically, per-point error had a median of 3.9 mm. Median errors by anatomy were 3.2 mm (feet), 4.3 mm (ear), and 5.3 mm (lower leg), with 5 mm coverage 92-95%, 84-90%, and 72-86%, respectively. Feet vs. lower leg differed significantly (Delta median ~1.1 mm; p<0.001). Conclusion: A depth-only, markerless AR pipeline on HMDs achieved ~3-4 mm median error across feet, ear, and lower leg in live surgical settings without fiducials, approaching typical clinical error thresholds for moderate-risk tasks. Human-guided initialization plus global-to-local registration enabled accurate alignment on small or low-curvature targets, improving the clinical readiness of markerless AR guidance.

Problem

• Developing markerless AR registration for surgical guidance without fiducials
• Evaluating accuracy across small or low-curvature anatomical structures
• Validating depth-only alignment method in real-life operative settings

Innovation

• Depth-only markerless AR registration on head-mounted display
• Human-guided initialization plus global-to-local registration
• Achieved millimeter accuracy across multiple anatomies without fiducials