CardioAI: A Multimodal AI-based System to Support Symptom Monitoring and Risk Detection of Cancer Treatment-Induced Cardiotoxicity

2024-10-06

arXiv.org

AI Summary

Cardiotoxicity induced by cancer therapeutics often progresses insidiously, posing significant challenges for early clinical detection and frequently leading to under-recognition. To address this, we propose a clinician–patient co-designed multimodal AI monitoring system that integrates continuous physiological signals from wearable devices with large language model–driven voice-based symptom elicitation, establishing a dual-domain (clinical and daily-life) dynamic surveillance paradigm. The system introduces a novel interpretable risk prediction module that concurrently generates quantitative risk scores and natural-language attribution summaries, ensuring deep alignment with clinical decision-making workflows. It incorporates temporal data fusion, eXplainable AI (XAI) modeling, and lightweight voice interaction technologies. Evaluated via heuristic assessment by four cardiologists and oncologists, the system demonstrates seamless integration into existing clinical workflows, effectively mitigating information overload while significantly improving the accuracy of early cardiotoxicity detection and timeliness of intervention.

Technology Category

• Computer Vision: Multi-modal Vision
• Humans and AI: Explainable AI (XAI) for Human Understanding
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Despite recent advances in cancer treatments that prolong patients' lives, treatment-induced cardiotoxicity remains one severe side effect. The clinical decision-making of cardiotoxicity is challenging, as non-clinical symptoms can be missed until life-threatening events occur at a later stage, and clinicians already have a high workload centered on the treatment, not the side effects. Our project starts with a participatory design study with 11 clinicians to understand their practices and needs; then we build a multimodal AI system, CardioAI, that integrates wearables and LLM-powered voice assistants to monitor multimodal non-clinical symptoms. Also, the system includes an explainable risk prediction module that can generate cardiotoxicity risk scores and summaries as explanations to support clinicians' decision-making. We conducted a heuristic evaluation with four clinical experts and found that they all believe CardioAI integrates well into their workflow, reduces their information overload, and enables them to make more informed decisions.

Problem

• Monitors non-clinical symptoms of cardiotoxicity using AI and wearables.
• Predicts cardiotoxicity risk with explainable AI to aid clinical decisions.
• Reduces clinician workload by integrating seamlessly into existing workflows.

Innovation

• Multimodal AI system integrates wearables and voice assistants
• Explainable risk prediction module generates cardiotoxicity risk scores
• Participatory design study informs clinician-centered AI development