Seeing Globally, Refining Locally: Global Visual Guidance and Local Ultrasound Cues for Robust Freehand 3-D Ultrasound Reconstruction

2026-07-14

AI Summary

Freehand 3D ultrasound reconstruction is highly susceptible to cumulative pose drift, particularly during long-trajectory scans. This work proposes a global–local fused pose estimation framework that leverages an external stereo camera for globally stable localization while incorporating anatomical structure-derived motion cues extracted from B-mode ultrasound images. A cross-modal feature fusion mechanism and multi-scale relative motion constraints are introduced to optimize poses in transformation space. By jointly integrating visual guidance with ultrasound anatomical information—a first in the field—the method significantly suppresses trajectory drift. Evaluated on the authors’ newly established FUSION-J and FUSION-L datasets, the approach reduces drift to 1.67 mm and 1.29 mm, respectively, representing improvements of 16.50% and 27.12% over the stereo-camera baseline. Furthermore, it achieves a Hausdorff distance of 1.58 mm in forearm artery reconstruction.

Abstract

Freehand 3-D ultrasound (US) imaging has attracted increasing attention owing to its intuitive volumetric visualization, ease of use, and low cost. However, accurate 3-D reconstruction critically depends on stable probe pose estimation, yet existing trackerless methods remain susceptible to accumulated pose errors, particularly over long scanning trajectories. To address this limitation, we propose a global-to-local pose estimation framework that exploits external camera observations for globally stable localization and B-mode US images for anatomy-aware local refinement. Specifically, the framework comprises a dual-camera branch that performs contextual feature aggregation across camera views and temporal observations to estimate a globally consistent probe trajectory, and a B-mode branch that performs anatomical feature aggregation from sequential US images to capture tissue-dependent local motion cues. A cross-modal fusion module subsequently integrates the contextual camera features and anatomical US features to predict pose residuals and refine the camera-derived estimates in the transformation space. Furthermore, a multi-scale pose loss constrains relative motion over multiple temporal horizons to suppress accumulated drift during extended scans. The proposed framework is validated on phantom and in vivo datasets. On two in-house datasets (FUSION-J and FUSION-L) collected using different machines, the proposed US + Dual-Cam model reduces average trajectory drift to 1.67 mm and 1.29 mm, representing improvement of 16.50% and 27.12%, respectively, over a strong dual-camera baseline, while substantially outperforming US-only pose estimation (>13 mm drift). In in vivo forearm arteries reconstruction, it achieves Hausdorff distances of 1.58 mm, demonstrating the effectiveness of the proposed method on real clinical scenarios.

Problem

• freehand 3D ultrasound
• pose estimation
• trajectory drift
• ultrasound reconstruction
• trackerless

Innovation

• global-to-local pose estimation
• cross-modal fusion
• freehand 3D ultrasound
• trajectory drift suppression
• anatomy-aware refinement