PRISM: Predictive Runtime In-place Scaling and Model Selection for Edge Microservices

2026-08-22

AI Summary

PRISM通过预测指导的运行时框架选择模型变体和CPU分配，以解决边缘AI服务中资源浪费问题，减少能耗并保持性能。

Abstract

Latency-sensitive edge AI services must balance strict deadlines, output quality, and limited compute and energy budgets. However, static CPU provisioning wastes resources because inference cost varies substantially across inputs, model variants, and runtime conditions. We present PRISM, a prediction-guided runtime framework that jointly selects model variants and CPU allocations for containerized edge microservices. Using container-level energy monitoring and lightweight regression models, PRISM adapts each pipeline stage in place and minimizes predicted CPU-package energy under deadline, resource, and offline model-level Quality of Result (QoR) constraints. We evaluate PRISM on more than 52,000 requests in an Automatic License Plate Recognition (ALPR) pipeline with detection and recognition stages. For detection, PRISM reduces energy consumption by 36 % compared to the strongest static configuration while preserving a comparable success rate and using less than half of the average CPU allocation. For recognition, it reaches near-static-best performance with lower average CPU allocation. These results show that predictive in-place adaptation is a practical mechanism for making time-sensitive AI microservice pipelines more energy-efficient at the edge.

Problem

• Latency-sensitive
• Edge AI services
• Static CPU provisioning
• Inference cost variation

Innovation

• Predictive Runtime
• In-place Scaling
• Model Selection
• Energy Efficiency
• Edge Microservices