Explaining Recovery Trajectories of Older Adults Post Lower-Limb Fracture Using Modality-wise Multiview Clustering and Large Language Models

2025-06-13

AI Summary

Identifying high-risk elderly patients with lower-limb fractures during home-based rehabilitation remains challenging due to heterogeneous, unstructured sensor data and lack of clinically interpretable patterns. Method: We propose an unsupervised, interpretable multimodal clustering framework integrating “modality-decoupled clustering” with context-aware large language model (LLM) prompting. This jointly models and semantically annotates heterogeneous wearable sensor streams—including accelerometry, step count, ambient motion, GPS, heart rate, and sleep—enabling automatic clinical interpretation of cluster structures. Contribution/Results: To our knowledge, this is the first work to achieve automated clinical semantic labeling of multimodal physiological–behavioral clusters. Cluster labels demonstrate statistically significant correlations with multiple validated clinical assessment scores (p < 0.01). Critically, the framework reliably identifies aberrant rehabilitation trajectories using only off-the-shelf wearable data—enabling timely, non-invasive early intervention without requiring manual annotation or clinical supervision.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Clustering
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search

Abstract

Interpreting large volumes of high-dimensional, unlabeled data in a manner that is comprehensible to humans remains a significant challenge across various domains. In unsupervised healthcare data analysis, interpreting clustered data can offer meaningful insights into patients' health outcomes, which hold direct implications for healthcare providers. This paper addresses the problem of interpreting clustered sensor data collected from older adult patients recovering from lower-limb fractures in the community. A total of 560 days of multimodal sensor data, including acceleration, step count, ambient motion, GPS location, heart rate, and sleep, alongside clinical scores, were remotely collected from patients at home. Clustering was first carried out separately for each data modality to assess the impact of feature sets extracted from each modality on patients' recovery trajectories. Then, using context-aware prompting, a large language model was employed to infer meaningful cluster labels for the clusters derived from each modality. The quality of these clusters and their corresponding labels was validated through rigorous statistical testing and visualization against clinical scores collected alongside the multimodal sensor data. The results demonstrated the statistical significance of most modality-specific cluster labels generated by the large language model with respect to clinical scores, confirming the efficacy of the proposed method for interpreting sensor data in an unsupervised manner. This unsupervised data analysis approach, relying solely on sensor data, enables clinicians to identify at-risk patients and take timely measures to improve health outcomes.

Problem

• Interpreting high-dimensional unlabeled sensor data for recovery insights
• Clustering multimodal data to understand lower-limb fracture recovery
• Validating LLM-generated cluster labels against clinical outcomes

Innovation

• Modality-wise multiview clustering for recovery analysis
• Large language models for cluster label inference
• Unsupervised sensor data interpretation with clinical validation