ClipGS-VR: Immersive and Interactive Cinematic Visualization of Volumetric Medical Data in Mobile Virtual Reality

2026-01-27

AI Summary

This work addresses the dual challenges of efficiency and visual fidelity in interactive medical volume visualization with arbitrary-angle slicing for mobile virtual reality. We propose an optimized framework based on 3D Gaussian Splatting that restructures the neural inference pipeline of ClipGS, integrates precomputed multi-view slice states into a unified rendering structure, and introduces a gradient-guided opacity modulation mechanism. Our approach achieves, for the first time, real-time high-fidelity arbitrary-angle slice rendering of medical volumes on consumer-grade mobile VR devices, delivering near offline-rendering quality while significantly enhancing interactivity and system usability.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Hardware-aware ML
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Security and Privacy: Data transparency and provenance

Abstract

High-fidelity cinematic medical visualization on mobile virtual reality (VR) remains challenging. Although ClipGS enables cross-sectional exploration via 3D Gaussian Splatting, it lacks arbitrary-angle slicing on consumer-grade VR headsets. To achieve real-time interactive performance, we introduce ClipGS-VR and restructure ClipGS's neural inference into a consolidated dataset, integrating high-fidelity layers from multiple pre-computed slicing states into a unified rendering structure. Our framework further supports arbitrary-angle slicing via gradient-based opacity modulation for smooth, visually coherent rendering. Evaluations confirm our approach maintains visual fidelity comparable to offline results while offering superior usability and interaction efficiency.

Problem

• cinematic visualization
• volumetric medical data
• mobile virtual reality
• arbitrary-angle slicing
• interactive visualization

Innovation

• Gaussian Splatting
• mobile VR
• volumetric visualization
• arbitrary-angle slicing
• real-time rendering