MANE: A Multi-Path Adaptive Network for Edge Onloading of Deep Neural Networks

2026-09-13

AI Summary

本文提出MANE框架，通过多路径尾部架构和三阶段训练方案解决边缘服务器在处理多设备推理时的负载管理和延迟问题。

Abstract

Split computing constitutes a widely used distributed inference approach, where a lightweight head model is onloaded onto the device and a heavier tail model resides on an edge server, leveraging the growing computational capabilities of modern System-on-Chips while alleviating server load. As intelligent indoor environments such as smart offices grow increasingly populated with diverse IoT devices, a single edge server must simultaneously assist multiple devices, each competing for the same shared inference resources. Without a principled mechanism to manage this shared load, the server is quickly overwhelmed, causing latency SLO violations and rendering server-assisted inference ineffective. In this work, we present MANE, a distributed inference framework that equips the server with a multi-path tail architecture, enabling a dynamic accuracy--throughput trade-off at runtime. By introducing a novel multi-path model architecture, a three-stage training scheme featuring a Joint Head Network Distillation loss and a hysteresis-based scheduler with an equitable device-fallback policy, MANE maintains over 80% SLO satisfaction rate where state-of-the-art onloading methods fail completely, while preserving accuracy 6pp higher than on-device alternatives, across up to 40 concurrent devices.

Problem

• split computing
• edge server
• shared inference resources
• latency SLO violations

Innovation

• multi-path tail architecture
• dynamic accuracy-throughput trade-off
• Joint Head Network Distillation loss
• hysteresis-based scheduler