DiLaServe: High SLO Attainment Serving for Diffusion Language Models

2026-06-27

AI Summary

Diffusion language models (DLMs) face a fundamental tension between meeting low-latency service-level objectives (SLOs) and achieving high throughput, primarily due to the speed-quality trade-off in confidence-based denoising, suboptimal parallelism selection under dynamic workloads, and non-uniform computational overhead introduced by approximate KV caching. To address these challenges, this work proposes DiLaServe—the first cluster-scale serving system tailored for DLMs—that jointly optimizes latency and throughput through deadline-aware scheduling, confidence-threshold-driven adaptive load control, and quality-aware dynamic cluster reconfiguration that accounts for the heterogeneity of approximate KV caches. Experimental results demonstrate that DiLaServe improves SLO compliance by up to 56.6 percentage points, reduces end-to-end latency by as much as 46%, and incurs less than 1% accuracy degradation across diverse benchmarks and real-world workloads.

Abstract

Diffusion language models (DLMs) have recently emerged as a promising alternative to conventional autoregressive language models. By generating multiple tokens in parallel during each denoising step, they offer higher inference throughput while maintaining competitive quality. However, realizing these throughput gains while meeting latency SLOs in a serving system requires addressing challenges introduced by DLMs' unique characteristics. These include navigating the speed-quality tradeoff created by confidence-based denoising, choosing appropriate parallelization levels across model instances under fluctuating load, and coordinating approximate KV caching mechanisms that introduce non-uniform per-step costs. To address these challenges, we present DiLaServe, a cluster-level serving system for DLMs. DiLaServe enables deadline-aware scheduling and adaptive load control through confidence-threshold adjustment, and dynamically reconfigures the cluster by solving a quality-aware optimization problem, while explicitly modeling the step-level heterogeneity introduced by approximate KV caching. Across multiple benchmarks and real-world traces, DiLaServe improves SLO attainment by up to 56.6 percentage points and reduces end-to-end request latency by up to 46\% while incurring less than 1\% accuracy drop.

Problem

• Diffusion Language Models
• SLO Attainment
• Latency
• Parallel Generation
• KV Caching

Innovation

• Diffusion Language Models
• SLO-aware Serving
• Confidence-based Denoising
• Approximate KV Caching
• Dynamic Cluster Reconfiguration