Diagnosing Shoulder Disorders Using Multimodal Large Language Models and Consumer-Grade Cameras

2025-10-10

AI Summary

To address the challenges of early diagnosis—namely, high cost, limited accessibility, and insufficient medical resources—for shoulder disorders in underserved regions, this paper proposes the Hybrid Motion Video Diagnosis framework (HMVDx). HMVDx decouples action understanding from clinical diagnosis and introduces a usability index evaluation system grounded in a medical decision-making chain. It employs consumer-grade cameras for motion video acquisition and leverages multimodal large language models (MLLMs) to jointly model action recognition and graded diagnostic inference. Experimental results demonstrate that HMVDx achieves 92.4% diagnostic accuracy for shoulder joint injuries—representing a 79.6% improvement over end-to-end video-based diagnosis baselines. These findings robustly validate the efficacy and clinical adaptability of low-barrier MLLMs for video-based medical interpretation in primary care settings.

Technology Category

• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Computer Vision: Video Understanding & Activity Analysis
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Shoulder disorders, such as frozen shoulder (a.k.a., adhesive capsulitis), are common conditions affecting the health of people worldwide, and have a high incidence rate among the elderly and workers engaged in repetitive shoulder tasks. In regions with scarce medical resources, achieving early and accurate diagnosis poses significant challenges, and there is an urgent need for low-cost and easily scalable auxiliary diagnostic solutions. This research introduces videos captured by consumer-grade devices as the basis for diagnosis, reducing the cost for users. We focus on the innovative application of Multimodal Large Language Models (MLLMs) in the preliminary diagnosis of shoulder disorders and propose a Hybrid Motion Video Diagnosis framework (HMVDx). This framework divides the two tasks of action understanding and disease diagnosis, which are respectively completed by two MLLMs. In addition to traditional evaluation indicators, this work proposes a novel metric called Usability Index by the logical process of medical decision-making (action recognition, movement diagnosis, and final diagnosis). This index evaluates the effectiveness of MLLMs in the medical field from the perspective of the entire medical diagnostic pathway, revealing the potential value of low-cost MLLMs in medical applications for medical practitioners. In experimental comparisons, the accuracy of HMVDx in diagnosing shoulder joint injuries has increased by 79.6% compared with direct video diagnosis, a significant technical contribution to future research on the application of MLLMs for video understanding in the medical field.

Problem

• Developing low-cost shoulder disorder diagnosis using consumer cameras
• Applying multimodal language models for medical video understanding
• Addressing diagnostic challenges in resource-scarce medical regions

Innovation

• Uses consumer-grade cameras for low-cost diagnosis
• Employs two MLLMs for action understanding and diagnosis
• Proposes Hybrid Motion Video Diagnosis framework HMVDx