Optimization and Application of Cloud-based Deep Learning Architecture for Multi-Source Data Prediction

2024-10-16

arXiv.org

AI Summary

To address low prediction accuracy and poor deployment efficiency in early diabetes risk assessment, this study develops a cloud-native deep learning system tailored for multi-source heterogeneous health data, deployed on AWS. Methodologically, it innovatively integrates Apache Airflow–based automated pipelines with GPU elastic scheduling to enable an end-to-end training–deployment闭环 (cycle time: 18.7 hours). A lightweight deep neural network architecture, specifically designed for diabetes prediction, achieves 89.8% accuracy, 92.3% sensitivity, and 95.1% specificity on real-world clinical data, yielding an overall predictive performance of 94.2%. The system supports real-time risk scoring and automated intervention triggering, contributing to a significant 37.5% reduction in diabetes incidence among the target population. This work establishes a reusable technical paradigm for cloud-based intelligent prevention and control of chronic diseases.

Technology Category

• Machine Learning: Hardware-aware ML
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Data Mining & Knowledge Management: Scalability, Parallel & Distributed Systems

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

This study develops a cloud-based deep learning system for early prediction of diabetes, leveraging the distributed computing capabilities of the AWS cloud platform and deep learning technologies to achieve efficient and accurate risk assessment. The system utilizes EC2 p3.8xlarge GPU instances to accelerate model training, reducing training time by 93.2% while maintaining a prediction accuracy of 94.2%. With an automated data processing and model training pipeline built using Apache Airflow, the system can complete end-to-end updates within 18.7 hours. In clinical applications, the system demonstrates a prediction accuracy of 89.8%, sensitivity of 92.3%, and specificity of 95.1%. Early interventions based on predictions lead to a 37.5% reduction in diabetes incidence among the target population. The system's high performance and scalability provide strong support for large-scale diabetes prevention and management, showcasing significant public health value.

Problem

• Cloud Computing
• Deep Learning
• Diabetes Prediction

Innovation

• Cloud Computing
• Deep Learning
• Diabetes Prediction