Budget-Adaptive Routing: Skipping the Weak When the Strong Answers Anyway

2026-06-29

AI Summary

This work addresses the inefficiency in existing edge-cloud collaborative inference systems, where a weak model must first perform forward computation before routing decisions are made under dynamic offloading budgets, leading to unnecessary resource consumption. To overcome this limitation, the authors propose a budget-adaptive lightweight routing mechanism that directly determines the routing path from raw input pixels, dynamically choosing either to bypass the weak model entirely or to utilize its output. The designed router incurs only 0.153 GFLOPs and, for the first time, enables automatic strategy switching based on the available offloading budget, thereby achieving the Pareto-optimal accuracy boundary across the full operational range. Experiments demonstrate that the proposed method surpasses the accuracy ceiling of fixed strategies on PASCAL VOC, reduces per-frame latency by up to 19.1 ms (approximately 30%), and attains a mean average precision (mAP) that exceeds the peak performance of the strong model by 1.7 percentage points at low computational overhead.

Abstract

Edge-cloud inference collaborations are often designed with a routing estimator that decides whether to offload each frame from weak models at the edge to stronger models in the cloud. Existing systems place the routing estimator after the weak detector, so the weak forward pass still runs even on frames that are later offloaded. In this paper, we argue that this weak-conditioned design can be suboptimal when the offload budget varies. First, we present a competitive weak-skipping estimator (0.153 GFLOPs, about 29x lighter than the weak detector at 4.49 GFLOPs) that extracts routing signal from raw pixels, outperforming the common after-weak placement weak-conditioned baselines. Second, we show that neither weak-skipping nor weak-conditioned placement dominates across the full operating curve, and we propose budget-adaptive routing, which selects between them by offload budget via two offline-tuned thresholds. On PASCAL VOC, our budget-adaptive router traces the upper accuracy envelope of both fixed placements across the operating range. Our method reduces per-frame latency by up to 19.1 ms (about 30% lower at rho = 0.9). Besides outperforming SOTA methods, it is surprisingly stronger than the strong model (+1.7 pp over the strong model's peak mAP) at some operating points with far less compute. Artifacts are available at https://github.com/ViGeng/bgt-ada

Problem

• edge-cloud inference
• routing estimator
• offload budget
• weak model skipping
• adaptive routing

Innovation

• budget-adaptive routing
• weak-skipping estimator
• edge-cloud inference
• offload budget
• model routing