ARGS: Auto-Regressive Gaussian Splatting via Parallel Progressive Next-Scale Prediction

2026-04-01

AI Summary

Existing 3D generation methods suffer from limitations in controllable detail, structural consistency, and computational efficiency. This work proposes Autoregressive Gaussian Splatting (ARGS), a novel framework that introduces autoregressive multiscale generation into 3D Gaussian representations for the first time. By organizing Gaussians into a hierarchical tree structure, ARGS achieves efficient generation with logarithmic complexity (𝒪(log n)) and incorporates a tree-based Transformer that enables leaf nodes to attend to their ancestors, thereby enhancing structural coherence. Combined with Gaussian simplification and inverse-simplification strategies alongside multiscale parallel prediction, the method produces 3D content with high visual fidelity and controllable levels of detail, significantly improving generation speed without compromising quality.

Abstract

Auto-regressive frameworks for next-scale prediction of 2D images have demonstrated strong potential for producing diverse and sophisticated content by progressively refining a coarse input. However, extending this paradigm to 3D object generation remains largely unexplored. In this paper, we introduce auto-regressive Gaussian splatting (ARGS), a framework for making next-scale predictions in parallel for generation according to levels of detail. We propose a Gaussian simplification strategy and reverse the simplification to guide next-scale generation. Benefiting from the use of hierarchical trees, the generation process requires only \(\mathcal{O}(\log n)\) steps, where \(n\) is the number of points. Furthermore, we propose a tree-based transformer to predict the tree structure auto-regressively, allowing leaf nodes to attend to their internal ancestors to enhance structural consistency. Extensive experiments demonstrate that our approach effectively generates multi-scale Gaussian representations with controllable levels of detail, visual fidelity, and a manageable time consumption budget.

Problem

• 3D generation
• auto-regressive modeling
• multi-scale prediction
• Gaussian splatting
• next-scale prediction

Innovation

• Auto-Regressive Generation
• Gaussian Splatting
• Multi-Scale 3D Generation
• Tree-Based Transformer
• Hierarchical Representation