DINO-CVA: A Multimodal Goal-Conditioned Vision-to-Action Model for Autonomous Catheter Navigation

2025-10-19

AI Summary

Cardiac catheterization remains highly operator-dependent, leading to procedural fatigue, excessive radiation exposure, and inter-operator variability in outcomes. To address the limited autonomy of existing robotic systems, this work proposes a multimodal, goal-conditioned vision–action transfer framework. It introduces the first joint embedding of intravascular visual observations and joystick kinematics, coupled with a goal-conditioning mechanism that enables anatomy-aware, interpretable, and controllable autonomous navigation. The method integrates behavioral cloning and autoregressive action prediction, fusing multimodal features and conditioning action generation on anatomical targets. Experiments on synthetic vascular phantoms demonstrate that the model achieves action prediction accuracy comparable to a kinematics-only baseline while establishing robust semantic associations between visual perception and control policies. These results validate the efficacy and clinical potential of the proposed multimodal autonomous navigation architecture.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Vision for Robotics & Autonomous Driving
• Natural Language Processing: Safety and Robustness

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Cardiac catheterization remains a cornerstone of minimally invasive interventions, yet it continues to rely heavily on manual operation. Despite advances in robotic platforms, existing systems are predominantly follow-leader in nature, requiring continuous physician input and lacking intelligent autonomy. This dependency contributes to operator fatigue, more radiation exposure, and variability in procedural outcomes. This work moves towards autonomous catheter navigation by introducing DINO-CVA, a multimodal goal-conditioned behavior cloning framework. The proposed model fuses visual observations and joystick kinematics into a joint embedding space, enabling policies that are both vision-aware and kinematic-aware. Actions are predicted autoregressively from expert demonstrations, with goal conditioning guiding navigation toward specified destinations. A robotic experimental setup with a synthetic vascular phantom was designed to collect multimodal datasets and evaluate performance. Results show that DINO-CVA achieves high accuracy in predicting actions, matching the performance of a kinematics-only baseline while additionally grounding predictions in the anatomical environment. These findings establish the feasibility of multimodal, goal-conditioned architectures for catheter navigation, representing an important step toward reducing operator dependency and improving the reliability of catheterbased therapies.

Problem

• Developing autonomous catheter navigation to reduce manual dependency
• Fusing visual and kinematic data for intelligent surgical guidance
• Creating goal-conditioned AI for precise catheter destination control

Innovation

• Multimodal fusion of vision and kinematics
• Goal-conditioned autoregressive action prediction
• Vision-aware and kinematic-aware joint embedding