Flow caching for autoregressive video generation

2026-02-11

AI Summary

Autoregressive video generation suffers from inefficiency due to its sequential, block-by-block processing, and existing caching methods struggle to accommodate the varying denoising requirements of different video blocks at the same timestep. This work proposes FlowCache—the first dynamic caching framework tailored for autoregressive video generation—that overcomes the limitations of conventional uniform caching assumptions through a block-aware independent caching strategy and a KV cache compression mechanism jointly optimized for importance and redundancy. Under fixed memory constraints, FlowCache achieves 2.38× and 6.7× speedups on MAGI-1 and SkyReels-V2, respectively, with negligible changes in VBench quality metrics (+0.87 / −0.79), significantly advancing autoregressive models toward efficient, high-quality generation of ultra-long videos.

Technology Category

• Computer Vision: Video Understanding & Activity Analysis
• Machine Learning: Deep Generative Models & Autoencoders
• Natural Language Processing: Generation

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Autoregressive models, often built on Transformer architectures, represent a powerful paradigm for generating ultra-long videos by synthesizing content in sequential chunks. However, this sequential generation process is notoriously slow. While caching strategies have proven effective for accelerating traditional video diffusion models, existing methods assume uniform denoising across all frames-an assumption that breaks down in autoregressive models where different video chunks exhibit varying similarity patterns at identical timesteps. In this paper, we present FlowCache, the first caching framework specifically designed for autoregressive video generation. Our key insight is that each video chunk should maintain independent caching policies, allowing fine-grained control over which chunks require recomputation at each timestep. We introduce a chunkwise caching strategy that dynamically adapts to the unique denoising characteristics of each chunk, complemented by a joint importance-redundancy optimized KV cache compression mechanism that maintains fixed memory bounds while preserving generation quality. Our method achieves remarkable speedups of 2.38 times on MAGI-1 and 6.7 times on SkyReels-V2, with negligible quality degradation (VBench: 0.87 increase and 0.79 decrease respectively). These results demonstrate that FlowCache successfully unlocks the potential of autoregressive models for real-time, ultra-long video generation-establishing a new benchmark for efficient video synthesis at scale. The code is available at https://github.com/mikeallen39/FlowCache.

Problem

• autoregressive video generation
• flow caching
• video synthesis
• generation speed
• chunkwise denoising

Innovation

• flow caching
• autoregressive video generation
• chunkwise caching
• KV cache compression
• efficient video synthesis