PolyPose: Localizing Deformable Anatomy in 3D from Sparse 2D X-ray Images using Polyrigid Transforms

2025-05-25

AI Summary

In interventional procedures, accurate 3D pose estimation of deformable anatomical structures from sparse intraoperative 2D X-ray images remains challenging. To address this, we propose a differentiable 2D/3D registration method based on polyrigid (multi-rigid) modeling. Our approach parameterizes soft-tissue and bone deformation as piecewise-rigid motion chains, explicitly incorporating anatomical motion priors while eliminating hand-crafted deformation regularization terms. By integrating a physically accurate X-ray projection model with gradient-based optimization, the method achieves robust registration between preoperative CT/MRI and intraoperative X-rays using only two fluoroscopic views. Evaluated on multi-center orthopedic and radiotherapy datasets, our method significantly outperforms state-of-the-art approaches—particularly under extremely sparse-view and limited-angle acquisition scenarios—demonstrating superior accuracy, robustness, and clinical applicability.

Technology Category

• Computer Vision: 3D Computer Vision
• Intelligent Robots: State Estimation
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics

Abstract

Determining the 3D pose of a patient from a limited set of 2D X-ray images is a critical task in interventional settings. While preoperative volumetric imaging (e.g., CT and MRI) provides precise 3D localization and visualization of anatomical targets, these modalities cannot be acquired during procedures, where fast 2D imaging (X-ray) is used instead. To integrate volumetric guidance into intraoperative procedures, we present PolyPose, a simple and robust method for deformable 2D/3D registration. PolyPose parameterizes complex 3D deformation fields as a composition of rigid transforms, leveraging the biological constraint that individual bones do not bend in typical motion. Unlike existing methods that either assume no inter-joint movement or fail outright in this under-determined setting, our polyrigid formulation enforces anatomically plausible priors that respect the piecewise rigid nature of human movement. This approach eliminates the need for expensive deformation regularizers that require patient- and procedure-specific hyperparameter optimization. Across extensive experiments on diverse datasets from orthopedic surgery and radiotherapy, we show that this strong inductive bias enables PolyPose to successfully align the patient's preoperative volume to as few as two X-ray images, thereby providing crucial 3D guidance in challenging sparse-view and limited-angle settings where current registration methods fail.

Problem

• Localizing 3D anatomy from sparse 2D X-rays
• Enabling deformable 2D/3D registration without bending bones
• Aligning preoperative volumes to few X-rays for 3D guidance

Innovation

• Uses polyrigid transforms for deformable registration
• Leverages piecewise rigid nature of human anatomy
• Aligns preoperative volume to sparse X-ray images