SSD: Spatially Speculative Decoding Accelerates Autoregressive Image Generation

2026-06-18

AI Summary

Autoregressive image generation suffers from inefficient inference due to the flattening of 2D images into 1D sequences, which disregards spatial locality. This work proposes a spatial speculative decoding framework that, for the first time, incorporates two-dimensional spatial correlations into autoregressive speculative decoding by simultaneously predicting tokens to the right and below the current position, thereby overcoming the limitations of conventional one-dimensional sequential prediction. By integrating discrete visual token modeling with a parallel token prediction mechanism, the method achieves up to a 13.3× speedup in inference while maintaining high-fidelity generation quality on the DPG-Bench and GenEval benchmarks.

Abstract

Autoregressive models excel in visual generation by treating images as 1D sequences of discrete tokens, mirroring language modeling. However, this flattening discards the intrinsic 2D spatial locality of visual signals, creating severe computational bottlenecks during inference. We introduce Spatially Speculative Decoding (SSD), a framework that aligns the predictive objective with the natural geometry of images. Rather than predicting only the immediate next token in a 1D sequence, our model simultaneously predicts the adjacent horizontal token and the token directly below it. By capitalizing on this 2D spatial correlation, spatially speculative decoding overcomes the memory wall in visual inference. Our approach accelerates autoregressive image generation by up to 13.3x while maintaining high fidelity on DPG-Bench and GenEval. Our results suggest that respecting the underlying geometry of vision unlocks massive computational efficiencies, paving the way for real-time, high-resolution autoregressive generative models.

Problem

• autoregressive image generation
• spatial locality
• computational bottleneck
• 2D spatial correlation
• visual inference

Innovation

• Spatially Speculative Decoding
• autoregressive image generation
• 2D spatial locality
• inference acceleration
• visual generative models