Problem
Research questions and friction points this paper is trying to address.
multi-tenant
Kubernetes
secure computing
AI workloads
supercomputing
Multi-tenant Kubernetes Use Cases for AI, Secure Computing and Data Services, and More
🤖 AI Summary
This work addresses the limitations of traditional single-tenant batch systems in meeting the demands of AI training, secure computation on sensitive data, and mixed workloads requiring flexibility and reproducibility. It presents the first deployment of a multi-tenant Kubernetes infrastructure on the HPE Cray EX supercomputer (Isambard-AI), integrating trusted research environments with distributed AI model hosting services. The proposed architecture leverages KubeRay, Ray, vLLM, and HPE Slingshot interconnects to deliver a sandboxed, persistent platform. By extending Kubernetes beyond conventional cloud environments into high-performance bare-metal systems, this study demonstrates the feasibility of scalable “Kubernetes-as-a-Service” for national-scale AI infrastructure. It further identifies key implementation challenges and outlines an evolutionary pathway, offering a co-design paradigm for multi-tenant confidential computing in domains such as healthcare.
📝 Abstract
Kubernetes, as a container orchestration engine, has been widely used in cloud-native ecosystems for several years. In supercomputing ecosystems, especially where bare-metal performance for compute and network devices are considered, the adoption is somewhat limited. However, with the increasing diversity of use cases such as AI, secure and confidential computing for sensitive data, and mixed workload orchestration, a traditional, single-tenant batch computing system does not offer the flexibility and reproducibility to which public cloud users are accustomed. Note that Kubernetes is not considered a replacement for batch scheduling systems, which have powerful features for large-scale MPI jobs with thousands of network end points. Rather, it is a complementary service provided as part of a national AI Research Resource. We evaluate Kubernetes deployment on a Hewlett Packard Enterprise (HPE) Cray EX supercomputerwith HPE Slingshot interconnect, called Isambard-AI, with co-design use cases. One is a Trusted Research Environment used for medical and health sciences. The other combines KubeRay, Ray, and vLLM to provide a distributed, sandboxed, persistent AI model hosting service targeting multi-tenant confidential computing. We discuss challenges and lessons learned, and where further development is needed to offer a production Kubernetes-as-a-Service on HPE Cray EX (and later) platforms.
Innovation
Methods, ideas, or system contributions that make the work stand out.
multi-tenant Kubernetes
confidential computing
AI model hosting
trusted research environment
HPE Cray EX
🔎 Similar Papers
No similar papers found.
