FoundationPose-Initialized 3D-2D Liver Registration for Surgical Augmented Reality

2026-02-19

AI Summary

This study addresses the challenges of insufficient tumor localization accuracy in laparoscopic liver surgery and the limitations of existing registration methods that rely heavily on organ contours and require complex finite element modeling. To overcome these issues, the authors propose a lightweight non-rigid registration framework that integrates laparoscopic depth maps with a baseline pose estimator to obtain an initial liver pose. Instead of conventional finite element models, the method employs non-rigid iterative closest point (NICP) for efficient deformation-based registration. Experiments on real patient data demonstrate an average target registration error of 9.91 mm across three cases, with the combined rigid-NICP approach significantly outperforming pure rigid registration. The proposed framework achieves clinically acceptable accuracy while substantially reducing engineering complexity and the need for specialized expertise.

Technology Category

• Intelligent Robots: Localization, Mapping, and Navigation
• Machine Learning: Calibration & Uncertainty Quantification
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments

Abstract

Augmented reality can improve tumor localization in laparoscopic liver surgery. Existing registration pipelines typically depend on organ contours; deformable (non-rigid) alignment is often handled with finite-element (FE) models coupled to dimensionality-reduction or machine-learning components. We integrate laparoscopic depth maps with a foundation pose estimator for camera-liver pose estimation and replace FE-based deformation with non-rigid iterative closest point (NICP) to lower engineering/modeling complexity and expertise requirements. On real patient data, the depth-augmented foundation pose approach achieved 9.91 mm mean registration error in 3 cases. Combined rigid-NICP registration outperformed rigid-only registration, demonstrating NICP as an efficient substitute for finite-element deformable models. This pipeline achieves clinically relevant accuracy while offering a lightweight, engineering-friendly alternative to FE-based deformation.

Problem

• 3D-2D registration
• liver surgery
• augmented reality
• non-rigid alignment
• surgical navigation

Innovation

• FoundationPose
• NICP
• 3D-2D registration
• surgical augmented reality
• liver surgery