Next Visual Granularity Generation

2025-08-18

AI Summary

This work addresses the challenge of multi-granularity modeling in image generation. We propose the Next Visual Granularity (NVG) generation framework, which decomposes an image into a structured sequence of tokens at identical spatial resolution but varying visual granularities, enabling hierarchical modeling—from global layout to local details—via coarse-to-fine progressive generation. Our method employs sequence-based modeling and trains cascaded class-conditional NVG models, demonstrating scalability on ImageNet. The core innovation is a controllable granularity transition mechanism that explicitly captures inter-granularity dependencies. Experiments show substantial improvements over the VAR series: on ImageNet, our method achieves FID scores of 3.03, 2.44, and 2.06 at three progressively finer scales, respectively. It yields higher-fidelity generations and exhibits greater stability under resolution scaling.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Natural Language Processing: Generation
• Machine Learning: Multimodal Learning

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

We propose a novel approach to image generation by decomposing an image into a structured sequence, where each element in the sequence shares the same spatial resolution but differs in the number of unique tokens used, capturing different level of visual granularity. Image generation is carried out through our newly introduced Next Visual Granularity (NVG) generation framework, which generates a visual granularity sequence beginning from an empty image and progressively refines it, from global layout to fine details, in a structured manner. This iterative process encodes a hierarchical, layered representation that offers fine-grained control over the generation process across multiple granularity levels. We train a series of NVG models for class-conditional image generation on the ImageNet dataset and observe clear scaling behavior. Compared to the VAR series, NVG consistently outperforms it in terms of FID scores (3.30 -> 3.03, 2.57 ->2.44, 2.09 -> 2.06). We also conduct extensive analysis to showcase the capability and potential of the NVG framework. Our code and models will be released.

Problem

• Generating images with hierarchical granularity control
• Decomposing images into structured multi-resolution sequences
• Progressive refinement from layout to fine details

Innovation

• Hierarchical image decomposition into structured granularity sequences
• Progressive refinement from global layout to fine details
• Multi-granularity control through iterative NVG generation framework