Problem
Research questions and friction points this paper is trying to address.
LLM inference
edge-side inference
cloud-based inference
server-grade GPUs
mobile devices
WiCi: Wireless GPU Computing Infrastructure
🤖 AI Summary
为解决移动设备上推理性能不足的问题，提出无线GPU计算基础设施WiCi，通过WiFi将GPU计算卸载到附近的服务器级GPU，提高推理效率和模型支持能力。
📝 Abstract
LLM inference applications are gaining significant traction. The demand for inference is growing exponentially, and the GPU usage of inference is increasingly surpassing that of training. Due to the mobility penalty, edge-side inference fails to deliver satisfactory performance. Consequently, most inference service providers currently rely on cloud-based inference, which incurs substantial, not sustainable costs for enterprises, and is even increasing in the agentic paradigm. Therefore, our goal is to enable powerful computing capabilities as server-grade GPUs on mobile devices. We propose Wireless GPU Computing Infrastructure (WiCi) in this paper. Through WiCi, mobile devices can wirelessly access server-grade GPUs, running inference tasks on mobile clients but offloading GPU-related computations to a nearby GPU via WiFi. WiCi introduces a series of designs to make sure the infrastructure is scalable with different applications, compatible with different mobile devices, and has comparable performance to running on a physical GPU. We test WiCi from mobile devices and find that WiCi can reduce time to first token by up to 90%, improve the token rate by approximately 39x compared to local inference on mobile devices for the same model, and support much larger models. WiCi also achieves up to nearly 80% of the native performance of the server-grade GPU across different applications.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Wireless GPU Computing
Offloading Computation
Server-Grade GPUs
Scalability
Mobile Inference
🔎 Similar Papers
No similar papers found.
