Generative Refinement Networks for Visual Synthesis

2026-04-14

AI Summary

This work addresses the limitations of diffusion models in computational efficiency and the constrained generation quality of autoregressive models caused by discretization and error accumulation. To overcome these challenges, the authors propose the Generative Refinement Network (GRN), which constructs a near-lossless latent space through hierarchical binary quantization, incorporates a human-like global refinement mechanism inspired by artistic painting practices, and introduces an entropy-guided sampling strategy enabling adaptive-step generation. GRN achieves state-of-the-art results on ImageNet with a reconstruction rFID of 0.56 and a class-conditional gFID of 1.81. Furthermore, it demonstrates exceptional performance in text-to-image and text-to-video generation tasks, significantly advancing both generation quality and efficiency.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Natural Language Processing: Generation

Application Category

• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems

Abstract

While diffusion models dominate the field of visual generation, they are computationally inefficient, applying a uniform computational effort regardless of different complexity. In contrast, autoregressive (AR) models are inherently complexity-aware, as evidenced by their variable likelihoods, but are often hindered by lossy discrete tokenization and error accumulation. In this work, we introduce Generative Refinement Networks (GRN), a next-generation visual synthesis paradigm to address these issues. At its core, GRN addresses the discrete tokenization bottleneck through a theoretically near-lossless Hierarchical Binary Quantization (HBQ), achieving a reconstruction quality comparable to continuous counterparts. Built upon HBQ's latent space, GRN fundamentally upgrades AR generation with a global refinement mechanism that progressively perfects and corrects artworks -- like a human artist painting. Besides, GRN integrates an entropy-guided sampling strategy, enabling complexity-aware, adaptive-step generation without compromising visual quality. On the ImageNet benchmark, GRN establishes new records in image reconstruction (0.56 rFID) and class-conditional image generation (1.81 gFID). We also scale GRN to more challenging text-to-image and text-to-video generation, delivering superior performance on an equivalent scale. We release all models and code to foster further research on GRN.

Problem

• visual synthesis
• computational inefficiency
• discrete tokenization
• error accumulation
• complexity-aware generation

Innovation

• Generative Refinement Networks
• Hierarchical Binary Quantization
• complexity-aware generation
• global refinement mechanism
• entropy-guided sampling