Fractal Generative Models

2025-02-24

AI Summary

This work addresses the limited modularity and lack of hierarchical abstraction in existing generative models. We propose the Fractal Generative Model (FGM), which employs autoregressive models as atomic generative units and recursively composes them to construct a self-similar, scale-invariant hierarchical architecture—the first instance of fractal structure integration at the foundational level of generative modeling. This design enables infinite recursive abstraction and cross-scale representation learning during generation. Evaluated on pixel-level image generation, FGM achieves significant improvements: +0.12 bits per dimension (bpd) in log-likelihood and an 18% reduction in Fréchet Inception Distance (FID), empirically validating the efficacy of fractal-based modularity. The implementation is publicly available.

Technology Category

• Machine Learning: Deep Generative Models & Autoencoders
• Natural Language Processing: Generation
• Computer Vision: Generative Adversarial Networks (GANs) for Vision

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization

Abstract

Modularization is a cornerstone of computer science, abstracting complex functions into atomic building blocks. In this paper, we introduce a new level of modularization by abstracting generative models into atomic generative modules. Analogous to fractals in mathematics, our method constructs a new type of generative model by recursively invoking atomic generative modules, resulting in self-similar fractal architectures that we call fractal generative models. As a running example, we instantiate our fractal framework using autoregressive models as the atomic generative modules and examine it on the challenging task of pixel-by-pixel image generation, demonstrating strong performance in both likelihood estimation and generation quality. We hope this work could open a new paradigm in generative modeling and provide a fertile ground for future research. Code is available at https://github.com/LTH14/fractalgen.

Problem

• Introducing fractal generative models
• Abstracting generative models into atomic modules
• Enhancing pixel-by-pixel image generation

Innovation

• Fractal generative models
• Atomic generative modules
• Recursive self-similar architecture