Normalizing Flows are Capable Generative Models

2024-12-09

arXiv.org

AI Summary

Normalized flows (NFs) remain underexploited for density estimation and generative modeling due to architectural complexity and limited scalability. This paper proposes TarFlow—a scalable NF architecture built upon a direction-alternating autoregressive Transformer that directly models pixel-level distributions within image patches. To enhance robustness and sample quality, we introduce Gaussian noise injection during training, post-training denoising, and a unified conditional/unconditional guidance mechanism. TarFlow is the first single-flow model to significantly surpass prior state-of-the-art methods on standard image likelihood estimation benchmarks, while simultaneously achieving sample fidelity and diversity on par with diffusion models. The implementation is publicly available.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Natural Language Processing: Generation

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems

Abstract

Normalizing Flows (NFs) are likelihood-based models for continuous inputs. They have demonstrated promising results on both density estimation and generative modeling tasks, but have received relatively little attention in recent years. In this work, we demonstrate that NFs are more powerful than previously believed. We present TarFlow: a simple and scalable architecture that enables highly performant NF models. TarFlow can be thought of as a Transformer-based variant of Masked Autoregressive Flows (MAFs): it consists of a stack of autoregressive Transformer blocks on image patches, alternating the autoregression direction between layers. TarFlow is straightforward to train end-to-end, and capable of directly modeling and generating pixels. We also propose three key techniques to improve sample quality: Gaussian noise augmentation during training, a post training denoising procedure, and an effective guidance method for both class-conditional and unconditional settings. Putting these together, TarFlow sets new state-of-the-art results on likelihood estimation for images, beating the previous best methods by a large margin, and generates samples with quality and diversity comparable to diffusion models, for the first time with a stand-alone NF model. We make our code available at https://github.com/apple/ml-tarflow.

Problem

• Enhancing Normalizing Flows for better generative modeling
• Improving sample quality in likelihood-based image generation
• Achieving state-of-the-art results with Transformer-based NF architecture

Innovation

• Transformer-based variant of Masked Autoregressive Flows
• Gaussian noise augmentation during training
• Post training denoising and effective guidance