VolSegGS: Segmentation and Tracking in Dynamic Volumetric Scenes via Deformable 3D Gaussians

2025-07-16

AI Summary

To address the challenge of interactive segmentation and tracking in large-scale dynamic volumetric data, this paper introduces the first real-time visualization analysis framework based on deformable 3D Gaussian splatting. The method innovatively employs deformable Gaussians as a compact spatiotemporal representation of dynamic volumetric scenes, integrating view-independent color modeling, an affinity field network, and an embedded multi-granularity segmentation mechanism to achieve efficient rendering and temporally coherent, continuous tracking. Compared with state-of-the-art approaches, our framework significantly improves segmentation accuracy and frame rate—achieving interactive performance (≥30 FPS)—across multiple time-varying scientific datasets. It also reduces GPU memory consumption by 42%–68% and training memory overhead by 53%, enabling exploratory analysis under resource-constrained conditions.

Technology Category

• Computer Vision: Segmentation
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Cognitive Modeling & Cognitive Systems: Affective Computing

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Security and Privacy: Data transparency and provenance
• Web Mining and Content Analysis: Web data visualization

Abstract

Visualization of large-scale time-dependent simulation data is crucial for domain scientists to analyze complex phenomena, but it demands significant I/O bandwidth, storage, and computational resources. To enable effective visualization on local, low-end machines, recent advances in view synthesis techniques, such as neural radiance fields, utilize neural networks to generate novel visualizations for volumetric scenes. However, these methods focus on reconstruction quality rather than facilitating interactive visualization exploration, such as feature extraction and tracking. We introduce VolSegGS, a novel Gaussian splatting framework that supports interactive segmentation and tracking in dynamic volumetric scenes for exploratory visualization and analysis. Our approach utilizes deformable 3D Gaussians to represent a dynamic volumetric scene, allowing for real-time novel view synthesis. For accurate segmentation, we leverage the view-independent colors of Gaussians for coarse-level segmentation and refine the results with an affinity field network for fine-level segmentation. Additionally, by embedding segmentation results within the Gaussians, we ensure that their deformation enables continuous tracking of segmented regions over time. We demonstrate the effectiveness of VolSegGS with several time-varying datasets and compare our solutions against state-of-the-art methods. With the ability to interact with a dynamic scene in real time and provide flexible segmentation and tracking capabilities, VolSegGS offers a powerful solution under low computational demands. This framework unlocks exciting new possibilities for time-varying volumetric data analysis and visualization.

Problem

• Enable interactive visualization of dynamic volumetric scenes
• Support real-time segmentation and tracking in 3D data
• Reduce computational demands for time-varying data analysis

Innovation

• Deformable 3D Gaussians for dynamic scenes
• Affinity field network for fine segmentation
• Embedded segmentation for continuous tracking