Xema: Efficient Diffusion Serving through Fine-Grained Memory Management and Auto-Configuration

2026-07-13

AI Summary

This work addresses the challenge of serving high-resolution image and long-video diffusion models under stringent GPU memory constraints, where existing memory optimization techniques incur substantial performance overhead. The authors propose a fine-grained, predictable memory optimization mechanism driven by tensor lifetimes: offline memory traces are generated via request templates to precisely apply mitigation strategies during critical memory-pressure intervals and establish a static tensor layout that minimizes fragmentation. An offline planner jointly optimizes parallelism, concurrency, and memory configuration to enable efficient deployment while satisfying service-level objectives (SLOs) and GPU memory limits. Experiments on Flux.2, CogVideoX-5B, and LTX-2 demonstrate up to a 3.7× improvement in SLO compliance and reduce planning time from 6.3 hours to 197 seconds.

Abstract

Diffusion models are increasingly deployed as production visual-generation services, where serving high-resolution image and long video generation is often limited by GPU memory. Popular memory-saving techniques such as weight offloading, sharding, and VAE slicing are often not practical because they tend to introduce significant performance overhead. In this paper, we present Xema, a memory-efficient diffusion serving system that exploits predictable tensor lifetimes for trace-guided memory optimization. For each request template, Xema derives an offline memory trace to identify short memory-pressure intervals and applies memory mitigation only within these intervals and only by the amount needed to fit the target GPU budget. Xema further constructs a static memory layout for tensors with predictable lifetimes, reducing fragmentation-induced reserved memory and making offline memory reasoning reliable at runtime. Built on this memory optimization layer, Xema introduces an offline planner that jointly selects parallelism, concurrency, and memory control under GPU memory and SLO constraints. The selected plan is stored in a plan table and directly used by the online serving runtime. We implement Xema on production diffusion pipelines and evaluate it with Flux.2, CogVideoX-5B, and LTX-2. Compared with existing serving configurations, Xema improves SLO attainment by up to 3.7x and reduces planning cost from 6.3 hours to 197 seconds compared with grid search.

Problem

• diffusion models
• GPU memory
• memory management
• serving system
• visual generation

Innovation

• fine-grained memory management
• trace-guided optimization
• static memory layout
• offline planning
• diffusion serving