EVER: Exact Volumetric Ellipsoid Rendering for Real-time View Synthesis

2024-10-02

arXiv.org

AI Summary

Existing 3D Gaussian Splatting (3DGS) rasterization methods suffer from popping artifacts, view-dependent density, and difficulty modeling lens effects (e.g., defocus blur, fisheye distortion). This paper introduces the first differentiable volumetric rendering framework based on ellipsoidal primitives, prioritizing emission. It eliminates discontinuity artifacts and decouples density from viewing direction via analytic ellipsoid-ray intersection, physically consistent volume integration, and alpha-free end-to-end optimization. The method enables real-time GPU-accelerated ray tracing, rendering 720p frames at ~30 FPS on an RTX 4090. On large-scale scenes such as Zip-NeRF, it achieves the sharpest reconstruction quality among current real-time approaches, significantly suppressing blending artifacts while naturally supporting complex lens effects.

Technology Category

• Computer Vision: 3D Computer Vision
• Intelligent Robots: Embodied AI
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Security and Privacy: Data transparency and provenance
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Personalized, context-aware and across-device search

Abstract

We present Exact Volumetric Ellipsoid Rendering (EVER), a method for real-time differentiable emission-only volume rendering. Unlike recent rasterization based approach by 3D Gaussian Splatting (3DGS), our primitive based representation allows for exact volume rendering, rather than alpha compositing 3D Gaussian billboards. As such, unlike 3DGS our formulation does not suffer from popping artifacts and view dependent density, but still achieves frame rates of $sim!30$ FPS at 720p on an NVIDIA RTX4090. Since our approach is built upon ray tracing it enables effects such as defocus blur and camera distortion (e.g. such as from fisheye cameras), which are difficult to achieve by rasterization. We show that our method is more accurate with fewer blending issues than 3DGS and follow-up work on view-consistent rendering, especially on the challenging large-scale scenes from the Zip-NeRF dataset where it achieves sharpest results among real-time techniques.

Problem

• Real-time differentiable emission-only volume rendering
• Eliminating popping artifacts and view-dependent density
• Enabling defocus blur and camera distortion effects

Innovation

• Exact volumetric ellipsoid rendering method
• Ray tracing enables defocus blur effects
• Real-time 30 FPS on NVIDIA RTX4090