iVR-GS: Inverse Volume Rendering for Explorable Visualization via Editable 3D Gaussian Splatting

2025-04-24

AI Summary

To address the high hardware overhead of traditional volume rendering and the inability of existing neural view synthesis (NVS) methods to support transfer-function (TF)-driven real-time scene editing, this paper proposes iVR-GS, an inverse volume rendering framework. iVR-GS introduces a novel TF-semantic disentanglement paradigm, decomposing TF mapping into composable and editable 3D Gaussian primitives to enable TF-guided dynamic visibility control and structural editing. Key technical innovations include TF-aware Gaussian initialization, multi-model divide-and-conquer optimization, and voxel-guided sampling—collectively enhancing both efficiency and fidelity. Quantitatively, iVR-GS achieves PSNR improvements of 2.1–4.7 dB over Plenoxels, CCNeRF, and vanilla 3DGS across multiple volumetric datasets. It supports millisecond-level TF response and Gaussian-level real-time editing, reduces GPU memory consumption by 63%, and enables high-frame-rate interactive visualization on a single GPU.

Technology Category

• Computer Vision: 3D Computer Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Virtualization and resource management in Web systems and infrastructures
• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Security and Privacy: Data transparency and provenance

Abstract

In volume visualization, users can interactively explore the three-dimensional data by specifying color and opacity mappings in the transfer function (TF) or adjusting lighting parameters, facilitating meaningful interpretation of the underlying structure. However, rendering large-scale volumes demands powerful GPUs and high-speed memory access for real-time performance. While existing novel view synthesis (NVS) methods offer faster rendering speeds with lower hardware requirements, the visible parts of a reconstructed scene are fixed and constrained by preset TF settings, significantly limiting user exploration. This paper introduces inverse volume rendering via Gaussian splatting (iVR-GS), an innovative NVS method that reduces the rendering cost while enabling scene editing for interactive volume exploration. Specifically, we compose multiple iVR-GS models associated with basic TFs covering disjoint visible parts to make the entire volumetric scene visible. Each basic model contains a collection of 3D editable Gaussians, where each Gaussian is a 3D spatial point that supports real-time scene rendering and editing. We demonstrate the superior reconstruction quality and composability of iVR-GS against other NVS solutions (Plenoxels, CCNeRF, and base 3DGS) on various volume datasets. The code is available at https://github.com/TouKaienn/iVR-GS.

Problem

• Enables interactive 3D volume exploration with editable Gaussian splatting
• Reduces rendering costs while allowing real-time scene editing
• Overcomes fixed visibility limitations in existing NVS methods

Innovation

• Uses 3D Gaussian splatting for inverse volume rendering
• Enables real-time scene rendering and editing
• Composes multiple models for full volumetric visibility