Motion-Guided Dual-Camera Tracker for Endoscope Tracking and Motion Analysis in a Mechanical Gastric Simulator

2024-03-08

AI Summary

To address the high cost, material sensitivity, and poor robustness of electromagnetic tracking in mechanical gastric simulators, this paper proposes a binocular vision-based endoscope tracking system for high-precision 3D pose estimation and motion analysis of flexible endoscope tips. Methodologically, we introduce cross-camera mutual template matching (CMT) to ensure geometric consistency between stereo views, and design a Mamba-based motion-guided head (MMH) that jointly encodes temporal motion priors and visual features to mitigate challenges including dynamic occlusion, appearance variation, and illumination distortion. Experimental results demonstrate that our system reduces mean and maximum pose estimation errors by 42% and 72%, respectively, compared to the second-best method. Moreover, it significantly enhances the separability of motion features between novice and expert operators. This work establishes a novel paradigm for objective and reliable endoscopic skill assessment.

Technology Category

• Computer Vision: Motion & Tracking
• Intelligent Robots: State Estimation
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Security and Privacy: Tracking, profiling, and countermeasures against them
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

Flexible endoscope motion tracking and analysis in mechanical simulators have proven useful for endoscopy training. Common motion tracking methods based on electromagnetic tracker are however limited by their high cost and material susceptibility. In this work, the motion-guided dual-camera vision tracker is proposed to provide robust and accurate tracking of the endoscope tip's 3D position. The tracker addresses several unique challenges of tracking flexible endoscope tip inside a dynamic, life-sized mechanical simulator. To address the appearance variation and keep dual-camera tracking consistency, the cross-camera mutual template strategy (CMT) is proposed by introducing dynamic transient mutual templates. To alleviate large occlusion and light-induced distortion, the Mamba-based motion-guided prediction head (MMH) is presented to aggregate historical motion with visual tracking. The proposed tracker achieves superior performance against state-of-the-art vision trackers, achieving 42% and 72% improvements against the second-best method in average error and maximum error. Further motion analysis involving novice and expert endoscopists also shows that the tip 3D motion provided by the proposed tracker enables more reliable motion analysis and more substantial differentiation between different expertise levels, compared with other trackers. Project page: https://github.com/PieceZhang/MotionDCTrack

Problem

• Develops a cost-effective endoscope tracking system
• Overcomes material susceptibility in electromagnetic trackers
• Enhances 3D motion analysis for endoscopy training

Innovation

• Motion-guided dual-camera vision tracker
• Cross-camera mutual template strategy (CMT)
• Mamba-based motion-guided prediction head (MMH)