HybridCollab: Unifying In-Person and Remote Collaboration for Cardiovascular Surgical Planning in Mobile Augmented Reality

2025-04-14

AI Summary

Traditional surgical planning for congenital heart disease relies on physical 3D-printed models, severely limiting collaborative engagement to co-located teams and precluding real-time remote participation. To address this, we present the first iOS-based augmented reality (AR) surgical planning system enabling hybrid (local + remote) collaboration. Leveraging ARKit, the system achieves high-fidelity spatial anchoring and synchronized multi-user real-time rendering. We pioneer the medical application of Apple’s GameKit framework to ensure consistent, low-latency cross-regional interaction. The system supports intuitive gesture-based manipulation, real-time annotation, and shared interactive model exploration. Deployed in clinical settings, it significantly improves preoperative communication efficiency and planning accuracy. Our work establishes a scalable, clinically validated technical paradigm for distributed surgical collaboration in telemedicine.

Technology Category

• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Humans and AI: Planning and Decision Support for Human-Machine Teams
• Multiagent Systems: Multiagent Planning

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Web applications in cross-disciplinary domains and verticals such as mixed reality, smart cities, and digital health
• Search and Retrieval-Augmented AI: Assisted, interactive, and conversational search
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization

Abstract

Surgical planning for congenital heart disease traditionally relies on collaborative group examinations of a patient's 3D-printed heart model, a process that lacks flexibility and accessibility. While mobile augmented reality (AR) offers a promising alternative with its portability and familiar interaction gestures, existing solutions limit collaboration to users in the same physical space. We developed HybridCollab, the first iOS AR application that introduces a novel paradigm that enables both in-person and remote medical teams to interact with a shared AR heart model in a single surgical planning session. For example, a team of two doctors in one hospital room can collaborate in real time with another team in a different hospital.Our approach is the first to leverage Apple's GameKit service for surgical planning, ensuring an identical collaborative experience for all participants, regardless of location. Additionally, co-located users can interact with the same anchored heart model in their shared physical space. By bridging the gap between remote and in-person collaboration across medical teams, HybridCollab has the potential for significant real-world impact, streamlining communication and enhancing the effectiveness of surgical planning. Watch the demo: https://youtu.be/hElqJYDuvLM.

Problem

• Enables real-time remote and in-person AR collaboration for surgical planning
• Overcomes limitations of traditional 3D-printed heart model examinations
• Unifies multi-location teams via shared interactive AR heart model

Innovation

• iOS AR app for hybrid surgical collaboration
• Uses Apple's GameKit for unified experience
• Shared AR heart model across locations