Hardware-Rasterized Ray-Based Gaussian Splatting

2025-03-24

AI Summary

This paper addresses the challenge that ray-based 3D Gaussian splatting (RayGS) struggles to balance rendering speed and quality under hardware rasterization, while suffering from aliasing due to MIP-level mismatch. We propose the first RayGS rendering framework fully compatible with standard GPU rasterization pipelines. Methodologically: (1) we introduce a geometry-driven ray–Gaussian integration model enabling efficient rasterization; (2) we design a MIP-aware anti-aliasing mechanism that unifies multi-scale sampling during both training and inference, eliminating aliasing artifacts. Our contributions are threefold: (i) the first real-time, high-fidelity novel-view synthesis solution based on RayGS; (ii) consistent superiority over existing RayGS methods in visual quality and rendering throughput across multiple benchmarks; and (iii) a practical pathway toward low-latency, high-fidelity applications such as VR and MR.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Hardware-aware ML
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

We present a novel, hardware rasterized rendering approach for ray-based 3D Gaussian Splatting (RayGS), obtaining both fast and high-quality results for novel view synthesis. Our work contains a mathematically rigorous and geometrically intuitive derivation about how to efficiently estimate all relevant quantities for rendering RayGS models, structured with respect to standard hardware rasterization shaders. Our solution is the first enabling rendering RayGS models at sufficiently high frame rates to support quality-sensitive applications like Virtual and Mixed Reality. Our second contribution enables alias-free rendering for RayGS, by addressing MIP-related issues arising when rendering diverging scales during training and testing. We demonstrate significant performance gains, across different benchmark scenes, while retaining state-of-the-art appearance quality of RayGS.

Problem

• Develops hardware rasterized rendering for 3D Gaussian Splatting
• Enables high-speed RayGS rendering for VR/MR applications
• Solves alias-free rendering via MIP-related scale divergence

Innovation

• Hardware rasterized rendering for RayGS
• Mathematically rigorous derivation for efficient rendering
• Alias-free rendering addressing MIP-related issues