When Gaussian Meets Surfel: Ultra-fast High-fidelity Radiance Field Rendering

2025-04-24

AI Summary

This work addresses the longstanding trade-off between real-time performance and visual fidelity in radiance field rendering, particularly the flickering and popping artifacts induced by viewpoint changes. We propose Gaussian-Enhanced Splats (GESs), a dual-scale representation: coarse-grain geometry and appearance are modeled via 2D opaque splats, while fine details are captured by nearby 3D Gaussian distributions—enabling sort-free, two-pass rasterization for real-time rendering. Our core contribution is the first tightly coupled dual-scale parameterization unifying splats and Gaussians, ensuring view-consistent, popping-free rendering. We further introduce practical variants—Mip-GES, Speedy-GES, Compact-GES, and 2D-GES—to support diverse deployment scenarios. Experiments demonstrate that GESs achieve state-of-the-art image quality while significantly accelerating rendering, eliminating viewpoint-transition artifacts, and natively supporting anti-aliasing, compression, and hardware acceleration.

Technology Category

• Computer Vision: 3D Computer Vision
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

We introduce Gaussian-enhanced Surfels (GESs), a bi-scale representation for radiance field rendering, wherein a set of 2D opaque surfels with view-dependent colors represent the coarse-scale geometry and appearance of scenes, and a few 3D Gaussians surrounding the surfels supplement fine-scale appearance details. The rendering with GESs consists of two passes -- surfels are first rasterized through a standard graphics pipeline to produce depth and color maps, and then Gaussians are splatted with depth testing and color accumulation on each pixel order independently. The optimization of GESs from multi-view images is performed through an elaborate coarse-to-fine procedure, faithfully capturing rich scene appearance. The entirely sorting-free rendering of GESs not only achieves very fast rates, but also produces view-consistent images, successfully avoiding popping artifacts under view changes. The basic GES representation can be easily extended to achieve anti-aliasing in rendering (Mip-GES), boosted rendering speeds (Speedy-GES) and compact storage (Compact-GES), and reconstruct better scene geometries by replacing 3D Gaussians with 2D Gaussians (2D-GES). Experimental results show that GESs advance the state-of-the-arts as a compelling representation for ultra-fast high-fidelity radiance field rendering.

Problem

• Develops bi-scale representation for radiance field rendering
• Achieves ultra-fast high-fidelity rendering without artifacts
• Enables anti-aliasing and compact storage extensions

Innovation

• Bi-scale GES representation for radiance fields
• Two-pass rendering with surfels and Gaussians
• Coarse-to-fine optimization for scene details