Semantic 3D Reconstructions with SLAM for Central Airway Obstruction

2025-09-16

AI Summary

To address the lack of real-time 3D reconstruction and semantic understanding during central airway obstruction (CAO) interventions—hindering robotic navigation—this paper proposes the first lightweight, real-time 3D reconstruction pipeline integrating semantic segmentation with monocular SLAM. Geometrically, it employs DROID-SLAM to generate dense point clouds; semantically, it couples a lightweight, airway-specific segmentation model to perform online semantic labeling of points via mask-guided refinement. The modular architecture enables cross-anatomical generalization. Evaluated on ex vivo airway phantoms, the reconstructed point cloud achieves a Chamfer distance of only 0.62 mm against CT ground truth—significantly improving real-time visualization and anatomical fidelity in critical regions over conventional methods. This work establishes an interpretable, spatially registered, semantics-augmented 3D map foundation for robot-assisted CAO intervention.

Technology Category

• Computer Vision: Segmentation
• Intelligent Robots: Localization, Mapping, and Navigation
• Planning, Routing, and Scheduling: Replanning and Plan Repair

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Central airway obstruction (CAO) is a life-threatening condition with increasing incidence, caused by tumors in and outside of the airway. Traditional treatment methods such as bronchoscopy and electrocautery can be used to remove the tumor completely; however, these methods carry a high risk of complications. Recent advances allow robotic interventions with lesser risk. The combination of robot interventions with scene understanding and mapping also opens up the possibilities for automation. We present a novel pipeline that enables real-time, semantically informed 3D reconstructions of the central airway using monocular endoscopic video. Our approach combines DROID-SLAM with a segmentation model trained to identify obstructive tissues. The SLAM module reconstructs the 3D geometry of the airway in real time, while the segmentation masks guide the annotation of obstruction regions within the reconstructed point cloud. To validate our pipeline, we evaluate the reconstruction quality using ex vivo models. Qualitative and quantitative results show high similarity between ground truth CT scans and the 3D reconstructions (0.62 mm Chamfer distance). By integrating segmentation directly into the SLAM workflow, our system produces annotated 3D maps that highlight clinically relevant regions in real time. High-speed capabilities of the pipeline allows quicker reconstructions compared to previous work, reflecting the surgical scene more accurately. To the best of our knowledge, this is the first work to integrate semantic segmentation with real-time monocular SLAM for endoscopic CAO scenarios. Our framework is modular and can generalize to other anatomies or procedures with minimal changes, offering a promising step toward autonomous robotic interventions.

Problem

• Real-time semantic 3D reconstruction of central airways
• Integration of segmentation with SLAM for obstruction identification
• Enabling automated robotic interventions for airway obstruction treatment

Innovation

• Combining DROID-SLAM with segmentation model
• Real-time semantic 3D airway reconstruction
• Integrating segmentation into SLAM workflow