STARFlow: Scaling Latent Normalizing Flows for High-resolution Image Synthesis

2025-06-06

AI Summary

This work addresses the scalability limitations of normalizing flow models for high-resolution image synthesis. We propose STARFlow, a latent-space-based scalable generative model centered on Transformer Autoregressive Flow (TARFlow). TARFlow integrates a depth-shallow hybrid Transformer architecture, pre-trained autoencoder-based latent space modeling, and a novel flow-guided training algorithm—enabling, for the first time, end-to-end maximum-likelihood training of normalizing flows at scale. STARFlow achieves performance on par with state-of-the-art diffusion models on both class-conditional and text-to-image high-resolution generation tasks, while preserving exact probabilistic modeling, invertibility, and continuous-domain likelihood optimization—overcoming longstanding bottlenecks in resolution and scalability inherent to conventional flow-based approaches.

Technology Category

• Machine Learning: Deep Generative Models & Autoencoders
• Computer Vision: Diffusion Models for Vision
• Natural Language Processing: Generation

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems

Abstract

We present STARFlow, a scalable generative model based on normalizing flows that achieves strong performance in high-resolution image synthesis. The core of STARFlow is Transformer Autoregressive Flow (TARFlow), which combines the expressive power of normalizing flows with the structured modeling capabilities of Autoregressive Transformers. We first establish the theoretical universality of TARFlow for modeling continuous distributions. Building on this foundation, we introduce several key architectural and algorithmic innovations to significantly enhance scalability: (1) a deep-shallow design, wherein a deep Transformer block captures most of the model representational capacity, complemented by a few shallow Transformer blocks that are computationally efficient yet substantially beneficial; (2) modeling in the latent space of pretrained autoencoders, which proves more effective than direct pixel-level modeling; and (3) a novel guidance algorithm that significantly boosts sample quality. Crucially, our model remains an end-to-end normalizing flow, enabling exact maximum likelihood training in continuous spaces without discretization. STARFlow achieves competitive performance in both class-conditional and text-conditional image generation tasks, approaching state-of-the-art diffusion models in sample quality. To our knowledge, this work is the first successful demonstration of normalizing flows operating effectively at this scale and resolution.

Problem

• Scaling normalizing flows for high-resolution image synthesis
• Combining normalizing flows with Autoregressive Transformers effectively
• Enhancing scalability via architectural and algorithmic innovations

Innovation

• Transformer Autoregressive Flow for expressive modeling
• Deep-shallow design enhances scalability efficiently
• Latent space modeling improves high-resolution synthesis