AnatomyCarve: A VR occlusion management technique for medical images based on segment-aware clipping

2025-07-07

AI Summary

To address poor visibility of target anatomical structures in 3D medical imaging caused by self-occlusion among tissues, this paper proposes a semantic segmentation-guided interactive volume cropping method for virtual reality (VR). The method dynamically identifies and selectively removes occluding structures in real time based on organ-level semantic segmentation, thereby exposing the region of interest while preserving full contextual information and intrinsic 3D spatial relationships. Innovatively, it formalizes the occlusion-handling logic from hand-drawn anatomical illustrations into a segment-aware cropping algorithm, seamlessly integrating direct volume rendering with natural gesture-based interaction to achieve photorealistic and interpretable visualization. User studies demonstrate high satisfaction among non-expert users; validation by neurosurgeons confirms its clinical utility in surgical planning, with additional potential for medical education and preoperative navigation.

Technology Category

• Computer Vision: Segmentation
• Humans and AI: Interaction Techniques and Devices
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Security and Privacy: Data transparency and provenance
• Systems and Infrastructure for Web, Mobile and WoT: Virtualization and resource management in Web systems and infrastructures
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics

Abstract

Visualizing 3D medical images is challenging due to self-occlusion, where anatomical structures of interest can be obscured by surrounding tissues. Existing methods, such as slicing and interactive clipping, are limited in their ability to fully represent internal anatomy in context. In contrast, hand-drawn medical illustrations in anatomy books manage occlusion effectively by selectively removing portions based on tissue type, revealing 3D structures while preserving context. This paper introduces AnatomyCarve, a novel technique developed for a VR environment that creates high-quality illustrations similar to those in anatomy books, while remaining fast and interactive. AnatomyCarve allows users to clip selected segments from 3D medical volumes, preserving spatial relations and contextual information. This approach enhances visualization by combining advanced rendering techniques with natural user interactions in VR. Usability of AnatomyCarve was assessed through a study with non-experts, while surgical planning effectiveness was evaluated with practicing neurosurgeons and residents. The results show that AnatomyCarve enables customized anatomical visualizations, with high user satisfaction, suggesting its potential for educational and clinical applications.

Problem

• Addresses self-occlusion in 3D medical image visualization
• Improves internal anatomy representation with selective clipping
• Enhances VR-based surgical planning and education

Innovation

• VR-based occlusion management for medical images
• Segment-aware clipping preserves spatial relations
• Combines advanced rendering with natural VR interactions