SepsisAI Orchestrator: A Containerized and Scalable Platform for Deploying AI Models and Real-Time Monitoring in Early Sepsis Detection
This study addresses key barriers to deploying clinical AI models—namely data heterogeneity, lack of deployment standardization, and stringent hospital requirements for high concurrency and low latency. The authors propose an open-source, modular platform integrating FHIR-inspired CDA preprocessing, NoSQL storage, containerized LightGBM model serving, and a Streamlit-based clinical dashboard, with scalable orchestration via Docker and Kubernetes. For the first time, they quantitatively demonstrate a U-shaped scaling behavior under clinical AI inference workloads: on a 12-thread CPU, increasing service replicas from 3 to 12 reduces p95 latency by 57.3% (from 3.3 s to 1.41 s) and eliminates request failures, whereas further scaling degrades performance due to scheduling contention. This work delivers an end-to-end, reproducible architecture for high-performance clinical AI deployment.