Memory-Efficient Training-Free Acceleration of Diffusion Transformers with BaryCache

2026-08-24

AI Summary

本文提出一种无训练加速方法，使用Barycentric Extrapolator减少Diffusion Transformers在图像和视频生成中的内存消耗并提高采样速度。

Abstract

Diffusion Transformers achieve high-fidelity image and video generation, but their iterative sampling remains expensive, for each denoising step requires large matrix operations. Existing cache-based acceleration reduces redundant computation yet increases the VRAM footprint by storing intermediate states, which can directly constrain inference batch size. In this work, we propose a training-free acceleration method that performs stepwise forecasting for DiT sampling using a Barycentric Extrapolator. By leveraging barycentric extrapolation, our predictor is numerically stable and alleviates oscillatory artifacts analogous to the Runge phenomenon during forward forecasting. Across extensive experiments on both image and video generation, our approach provides a favorable trade-off between memory usage and perceptual quality, while delivering up to 3.30x end-to-end sampling speedup compared with baseline DiT inference.

Problem

• Diffusion Transformers
• iterative sampling
• matrix operations
• VRAM footprint
• inference batch size

Innovation

• Barycentric Extrapolator
• Training-Free Acceleration
• Memory-Efficient
• Denoising Step
• Perceptual Quality