Breast-Rehab: A Postoperative Breast Cancer Rehabilitation Training Assessment System Based on Human Action Recognition

2025-12-11

AI Summary

Low adherence to home-based upper-limb rehabilitation among post-mastectomy breast cancer patients, coupled with the high hardware cost and poor clinical deployability of existing virtual reality solutions, poses a significant barrier to effective recovery. To address this, we propose a low-cost, lightweight home rehabilitation training and assessment system. Our method introduces a novel action segmentation algorithm that fuses visual and 3D skeletal data, and a domain-knowledge-enhanced Retrieval-Augmented Generation (RAG) framework that drives a multimodal large language model to generate clinically credible evaluation reports—substantially reducing computational requirements and hallucination risks. The system integrates a custom lightweight action recognition model, a WeChat Mini-Program for patients, and a nurse-facing management dashboard. Preliminary clinical validation demonstrates an average exercise frequency of 0.59 sessions per day over two weeks, confirming high feasibility and user acceptability. This work provides a scalable, resource-efficient technical pathway for precision rehabilitation in resource-constrained settings.

Technology Category

• Computer Vision: Low Level & Physics-based Vision
• Intelligent Robots: Manipulation
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Systems and Infrastructure for Web, Mobile and WoT: Virtualization and resource management in Web systems and infrastructures
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Postoperative upper limb dysfunction is prevalent among breast cancer survivors, yet their adherence to at-home rehabilitation exercises is low amidst limited nursing resources. The hardware overhead of commonly adopted VR-based mHealth solutions further hinders their widespread clinical application. Therefore, we developed Breast-Rehab, a novel, low-cost mHealth system to provide patients with out-of-hospital upper limb rehabilitation management. Breast-Rehab integrates a bespoke human action recognition algorithm with a retrieval-augmented generation (RAG) framework. By fusing visual and 3D skeletal data, our model accurately segments exercise videos recorded in uncontrolled home environments, outperforming standard models. These segmented clips, combined with a domain-specific knowledge base, guide a multi-modal large language model to generate clinically relevant assessment reports. This approach significantly reduces computational overhead and mitigates model hallucinations. We implemented the system as a WeChat Mini Program and a nurse-facing dashboard. A preliminary clinical study validated the system's feasibility and user acceptance, with patients achieving an average exercise frequency of 0.59 sessions/day over a two-week period. This work thus presents a complete, validated pipeline for AI-driven, at-home rehabilitation monitoring.

Problem

• Addresses low adherence to at-home rehabilitation exercises for breast cancer survivors
• Reduces hardware overhead and computational cost of mHealth solutions
• Provides accurate AI-driven monitoring and assessment in uncontrolled home environments

Innovation

• Fuses visual and 3D skeletal data for action recognition
• Combines segmented clips with knowledge base via RAG framework
• Implements system as low-cost WeChat Mini Program and dashboard