SparSTAR: Sparse Attention for SpaceTime AutoRegressive Video Synthesis

2026-08-11

AI Summary

This work addresses the high computational cost of attention in high-resolution, multi-scale autoregressive video generation with the InfinityStar model and the insufficient reliability of existing sparse attention patterns. To this end, we propose a training-free dynamic block-sparse attention mechanism that independently selects critical blocks for each expensive scale and attention head, thereby avoiding the quality degradation caused by reusing sparse masks across scales. The method efficiently computes attention along a forward sparse path and integrates seamlessly with a multi-scale image–video pyramid architecture. Evaluated on 720p text- or image-to-video generation tasks, our approach achieves approximately 1.6× end-to-end speedup over dense InfinityStar while preserving comparable VBench scores and reconstruction fidelity.

Abstract

InfinityStar extends visual autoregressive generation to video through a sequence of image and clip pyramids. Its changing scale and cross-clip context, however, leave late-scale attention costly and make sparse patterns reused from diffusion or image VAR models unreliable. We introduce SparSTAR, a training-free block-sparse attention method tailored to this setting. At each expensive scale and attention head, SparSTAR scores contiguous key blocks from the current query and key activations, retains required conditioning context, and executes the selected blocks through a forward-only sparse path. We analyze cross-scale consistency within a clip, pattern persistence across clip boundaries, and quality degradation as reuse spans increasingly distant scales. Across these analyses, important key blocks shift, showing that recomputing block selection at each target scale is more reliable than reusing a transferred mask. On 720p text-to-video and image-to-video generation, SparSTAR preserves every token and refinement scale while providing about a 1.6x end-to-end speedup and maintaining VBench and paired-output reconstruction fidelity close to dense InfinityStar.

Problem

• sparse attention
• video synthesis
• autoregressive generation
• computational efficiency
• multi-scale modeling

Innovation

• sparse attention
• autoregressive video synthesis
• block-sparse
• training-free
• multi-scale generation