RISE: Relay Inference and Online Scheduling for Efficient Edge-Device Collaborative Diffusion Model Services

2026-06-15

AI Summary

Existing edge deployment approaches struggle to balance generation quality and inference latency and lack effective edge-device collaborative denoising mechanisms. This work proposes RISE, a novel training-free relay inference framework that leverages the shared latent space among diffusion models within the same family. After a large edge model generates the semantic structure, RISE transfers intermediate latent representations to a compact device-side model for fine-grained refinement. Additionally, RISE incorporates a context-aware online scheduler based on multi-armed bandit algorithms to dynamically adapt to multidimensional runtime conditions. Experimental results demonstrate that RISE achieves up to 2.1× speedup on two benchmarks while preserving the generation quality of full-scale models and effectively balancing latency and fidelity under heterogeneous workloads.

Abstract

Text-to-image diffusion models are increasingly deployed at the network edge to serve heterogeneous workloads with diverse quality and latency requirements. However, existing deployment strategies choose either large edge-side models with high fidelity but high latency or lightweight device-side models that offer speed at the cost of semantic coherence. Moreover, these approaches rarely split the denoising workload between models of different sizes across edge servers and user devices. To bridge this gap, we propose RISE, a method for edge-device diffusion model services that combines relay inference with online scheduling. Driven by the finding that the latent intensity exhibits minimal deviation after a model handoff, RISE uses a training-free relay mechanism that exploits the shared latent space within a model family: the large model on the edge handles the early denoising steps that shape semantic structure, then passes the intermediate latent to a small device-side model for detail refinement. To deploy this mechanism as a practical service, a contextual bandit scheduler selects the best relay configuration based on prompt complexity, user preferences, network quality and real-time node loads. Experiments on two benchmarks show that RISE's relay mechanism achieves up to 2.1$\times$ speedup while preserving full-model quality, and its context-aware scheduler effectively balances quality and latency under mixed workloads.

Problem

• diffusion models
• edge computing
• latency-quality tradeoff
• model collaboration
• denoising workload splitting

Innovation

• relay inference
• edge-device collaboration
• diffusion models
• online scheduling
• latent space sharing