Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesis

2024-12-05

arXiv.org

AI Summary

To address the limitations of visual autoregressive (VAR) models—namely, constrained resolution, insufficient photorealism, and low inference efficiency in high-resolution image generation—this paper introduces Infinity, the first text-to-image (T2I) framework based on bit-level autoregressive modeling. Our approach features three key innovations: (1) a novel infinite-vocabulary tokenizer and classifier, theoretically supporting a countably infinite lexicon; (2) a bit-level self-correction decoding mechanism to enhance modeling fidelity; and (3) coordinated scaling of Transformer capacity to overcome fundamental bottlenecks in conventional VAR architectures. Evaluated on GenEval and ImageReward benchmarks, Infinity achieves scores of 0.73 and 0.96, respectively—surpassing SD3-Medium. It generates 1024×1024 images in just 0.8 seconds, representing a 2.6× speedup over prior autoregressive T2I methods and establishing a new state-of-the-art in autoregressive image synthesis.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Generation

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

We present Infinity, a Bitwise Visual AutoRegressive Modeling capable of generating high-resolution, photorealistic images following language instruction. Infinity redefines visual autoregressive model under a bitwise token prediction framework with an infinite-vocabulary tokenizer&classifier and bitwise self-correction mechanism, remarkably improving the generation capacity and details. By theoretically scaling the tokenizer vocabulary size to infinity and concurrently scaling the transformer size, our method significantly unleashes powerful scaling capabilities compared to vanilla VAR. Infinity sets a new record for autoregressive text-to-image models, outperforming top-tier diffusion models like SD3-Medium and SDXL. Notably, Infinity surpasses SD3-Medium by improving the GenEval benchmark score from 0.62 to 0.73 and the ImageReward benchmark score from 0.87 to 0.96, achieving a win rate of 66%. Without extra optimization, Infinity generates a high-quality 1024x1024 image in 0.8 seconds, making it 2.6x faster than SD3-Medium and establishing it as the fastest text-to-image model. Models and codes will be released to promote further exploration of Infinity for visual generation and unified tokenizer modeling.

Problem

• Scaling autoregressive modeling for high-resolution image synthesis
• Improving generation capacity with bitwise token prediction
• Achieving faster and superior text-to-image generation performance

Innovation

• Bitwise token prediction with infinite-vocabulary tokenizer
• Bitwise self-correction mechanism enhances generation details
• Scaling transformer size with infinite tokenizer vocabulary