Volume Encoding Gaussians: Transfer Function-Agnostic 3D Gaussians for Volume Rendering

2025-04-17

AI Summary

To address the challenge of interactive visualization of unstructured volumetric data generated by high-performance computing, this paper proposes VEG—the first 3D Gaussian-based representation for unstructured volume data. VEG decouples scalar field modeling from rendering appearance, enabling real-time volume rendering with arbitrary transfer functions. Its key innovations include: (i) a transfer-function-agnostic Gaussian volume encoding scheme, initialized directly from raw volumetric data without requiring structure-from-motion; (ii) an opacity-guided differentiable rendering training strategy supporting multiple transfer functions; and (iii) explicit local geometric modeling via adaptive Gaussian scaling and rotation. Experiments demonstrate that VEG achieves high-fidelity reconstruction across diverse scientific volumetric datasets, attaining a compression ratio of 3600× while sustaining >60 FPS interactive rendering on consumer-grade GPUs.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Security and Privacy: Data transparency and provenance
• Systems and Infrastructure for Web, Mobile and WoT: Data management and stream processing for Web, mobile and wireless applications

Abstract

While HPC resources are increasingly being used to produce adaptively refined or unstructured volume datasets, current research in applying machine learning-based representation to visualization has largely ignored this type of data. To address this, we introduce Volume Encoding Gaussians (VEG), a novel 3D Gaussian-based representation for scientific volume visualization focused on unstructured volumes. Unlike prior 3D Gaussian Splatting (3DGS) methods that store view-dependent color and opacity for each Gaussian, VEG decouple the visual appearance from the data representation by encoding only scalar values, enabling transfer-function-agnostic rendering of 3DGS models for interactive scientific visualization. VEG are directly initialized from volume datasets, eliminating the need for structure-from-motion pipelines like COLMAP. To ensure complete scalar field coverage, we introduce an opacity-guided training strategy, using differentiable rendering with multiple transfer functions to optimize our data representation. This allows VEG to preserve fine features across the full scalar range of a dataset while remaining independent of any specific transfer function. Each Gaussian is scaled and rotated to adapt to local geometry, allowing for efficient representation of unstructured meshes without storing mesh connectivity and while using far fewer primitives. Across a diverse set of data, VEG achieve high reconstruction quality, compress large volume datasets by up to 3600x, and support lightning-fast rendering on commodity GPUs, enabling interactive visualization of large-scale structured and unstructured volumes.

Problem

• Representing unstructured volume data with 3D Gaussians for visualization
• Enabling transfer-function-agnostic rendering in 3D Gaussian Splatting models
• Compressing large volume datasets while preserving fine features

Innovation

• VEG encodes scalar values for transfer-function-agnostic rendering
• Opacity-guided training optimizes data representation coverage
• Scaled and rotated Gaussians adapt to local geometry