FaaSMoE: A Serverless Framework for Multi-Tenant Mixture-of-Experts Serving

2026-04-29

AI Summary

This work addresses the inefficiency of deploying Mixture-of-Experts (MoE) models in multi-tenant settings, where keeping all experts resident leads to significant memory waste. The authors propose the first function-as-a-service (FaaS)-based MoE inference architecture for multi-tenant workloads, decoupling experts into stateless functions that are activated on demand with zero cold-start overhead. By separating the control plane from the execution plane and enabling configurable expert granularity, the design effectively balances elasticity against invocation overhead. Experimental results demonstrate that, under multi-tenant workloads using Qwen1.5-MoE-2.7B, the proposed approach reduces resource consumption to less than one-third compared to full-model deployment.

Abstract

Mixture-of-Experts (MoE) models offer high capacity with efficient inference cost by activating a small subset of expert models per input. However, deploying MoE models requires all experts to reside in memory, creating a gap between the resource used by activated experts and the provisioned resources. This underutilization is further pronounced in multi-tenant scenarios. In this paper, we propose FaaSMoE, a multi-tenant MoE serving architecture built on Function-as-a-Service (FaaS) platforms. FaaSMoE decouples the control and execution planes of MoE by deploying experts as stateless FaaS functions, enabling on-demand and scale-to-zero expert invocation across tenants. FaaSMoE further supports configurable expert granularity within functions, trading off per-expert elasticity for reduced invocation overhead. We implement a prototype with an open-source edge-oriented FaaS platform and evaluate it using Qwen1.5-moe-2.7B under multi-tenant workloads. Compared to a full-model baseline, FaaSMoE uses less than one third of the resources, demonstrating a practical and resource-efficient path towards scalable MoE serving in a multi-tenant environment.

Problem

• Mixture-of-Experts
• multi-tenant
• resource underutilization
• serverless
• MoE serving

Innovation

• Mixture-of-Experts
• Function-as-a-Service
• multi-tenant serving
• scale-to-zero
• resource efficiency