TexGS-VolVis: Expressive Scene Editing for Volume Visualization via Textured Gaussian Splatting

2025-07-17

AI Summary

Existing volume visualization methods rely on predefined rules for non-photorealistic rendering, resulting in limited stylistic diversity and poor flexibility; moreover, conventional 3D Gaussian primitives tightly couple geometry and appearance, hindering high-fidelity stylization and fine-grained editing. To address these limitations, we propose Textured Gaussian Lattice—a novel representation that decouples geometry from appearance by introducing 2D parameterized textures and spatially varying shading attributes, enabling geometry-consistent style transfer and controllable illumination modeling. Our method integrates differentiable Gaussian rendering, pretrained multimodal large models, and 2D-to-3D segmentation techniques into an end-to-end framework supporting text- or image-driven local editing and real-time rendering. Extensive evaluation on multiple volumetric datasets demonstrates significant improvements in rendering efficiency, visual quality, and interactive editing flexibility.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Security and Privacy: Data transparency and provenance
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Advancements in volume visualization (VolVis) focus on extracting insights from 3D volumetric data by generating visually compelling renderings that reveal complex internal structures. Existing VolVis approaches have explored non-photorealistic rendering techniques to enhance the clarity, expressiveness, and informativeness of visual communication. While effective, these methods often rely on complex predefined rules and are limited to transferring a single style, restricting their flexibility. To overcome these limitations, we advocate the representation of VolVis scenes using differentiable Gaussian primitives combined with pretrained large models to enable arbitrary style transfer and real-time rendering. However, conventional 3D Gaussian primitives tightly couple geometry and appearance, leading to suboptimal stylization results. To address this, we introduce TexGS-VolVis, a textured Gaussian splatting framework for VolVis. TexGS-VolVis employs 2D Gaussian primitives, extending each Gaussian with additional texture and shading attributes, resulting in higher-quality, geometry-consistent stylization and enhanced lighting control during inference. Despite these improvements, achieving flexible and controllable scene editing remains challenging. To further enhance stylization, we develop image- and text-driven non-photorealistic scene editing tailored for TexGS-VolVis and 2D-lift-3D segmentation to enable partial editing with fine-grained control. We evaluate TexGS-VolVis both qualitatively and quantitatively across various volume rendering scenes, demonstrating its superiority over existing methods in terms of efficiency, visual quality, and editing flexibility.

Problem

• Enabling arbitrary style transfer in volume visualization
• Decoupling geometry and appearance for better stylization
• Achieving flexible and controllable scene editing

Innovation

• Textured Gaussian splatting for volume visualization
• 2D Gaussian primitives with texture attributes
• Image- and text-driven scene editing