Sparse Voxels Rasterization: Real-time High-fidelity Radiance Field Rendering

2024-12-05

arXiv.org

AI Summary

To address the strong dependency on neural networks or 3D Gaussians, severe flickering artifacts, and low efficiency in real-time high-fidelity radiance field rendering, this paper proposes a purely explicit, neuron-free adaptive sparse voxel rasterization framework. Our key contributions are: (1) the first ray-direction-aware Morton encoding, eliminating flickering artifacts inherent in conventional Gaussian splatting; (2) an explicit multi-scale voxel allocation scheme coupled with depth-aware voxel sorting, achieving both geometric fidelity and rendering efficiency at an equivalent resolution of 65,536³; and (3) native support for downstream 3D reconstruction tasks—including volume fusion and Marching Cubes. Experiments demonstrate that our method outperforms the best existing neuron-free voxel-based approaches by >4 dB in PSNR and >10× in frame rate, while attaining state-of-the-art novel-view synthesis quality.

Technology Category

• Computer Vision: 3D Computer Vision
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

We propose an efficient radiance field rendering algorithm that incorporates a rasterization process on adaptive sparse voxels without neural networks or 3D Gaussians. There are two key contributions coupled with the proposed system. The first is to adaptively and explicitly allocate sparse voxels to different levels of detail within scenes, faithfully reproducing scene details with $65536^3$ grid resolution while achieving high rendering frame rates. Second, we customize a rasterizer for efficient adaptive sparse voxels rendering. We render voxels in the correct depth order by using ray direction-dependent Morton ordering, which avoids the well-known popping artifact found in Gaussian splatting. Our method improves the previous neural-free voxel model by over 4db PSNR and more than 10x FPS speedup, achieving state-of-the-art comparable novel-view synthesis results. Additionally, our voxel representation is seamlessly compatible with grid-based 3D processing techniques such as Volume Fusion, Voxel Pooling, and Marching Cubes, enabling a wide range of future extensions and applications.

Problem

• Efficient radiance field rendering
• High-fidelity scene reproduction
• Real-time adaptive sparse voxels

Innovation

• Sparse voxels rasterization technique
• Ray direction-dependent Morton ordering
• High-resolution grid without neural networks