TomoGraphView: 3D Medical Image Classification with Omnidirectional Slice Representations and Graph Neural Networks

2025-11-12

AI Summary

3D medical image classification faces dual challenges: complex volumetric spatial relationships and difficulty in modeling long-range dependencies, compounded by the scarcity of large-scale multimodal 3D datasets. Existing orthogonal-slice-based 2D transfer methods suffer from directional bias and loss of inter-slice spatial coherence. To address these issues, we propose an omnidirectional-slice–graph neural network (GNN) fusion framework. Our method introduces, for the first time, an omnidirectional volumetric sampling strategy that captures slices along arbitrary orientations, and constructs a spherical graph structure to enable topology-aware aggregation of slice features—explicitly encoding 3D spatial relationships. Leveraging a 2D vision foundation model as backbone, the framework integrates omnidirectional slicing, spherical graph modeling, and GNN-based feature fusion. It achieves significant performance gains over state-of-the-art slice-aggregation baselines across multiple 3D medical image classification tasks. We publicly release an open-source, reusable omnidirectional slicing toolkit.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Graph-based Machine Learning
• Reasoning under Uncertainty: Graphical Models

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

The growing number of medical tomography examinations has necessitated the development of automated methods capable of extracting comprehensive imaging features to facilitate downstream tasks such as tumor characterization, while assisting physicians in managing their growing workload. However, 3D medical image classification remains a challenging task due to the complex spatial relationships and long-range dependencies inherent in volumetric data. Training models from scratch suffers from low data regimes, and the absence of 3D large-scale multimodal datasets has limited the development of 3D medical imaging foundation models. Recent studies, however, have highlighted the potential of 2D vision foundation models, originally trained on natural images, as powerful feature extractors for medical image analysis. Despite these advances, existing approaches that apply 2D models to 3D volumes via slice-based decomposition remain suboptimal. Conventional volume slicing strategies, which rely on canonical planes such as axial, sagittal, or coronal, may inadequately capture the spatial extent of target structures when these are misaligned with standardized viewing planes. Furthermore, existing slice-wise aggregation strategies rarely account for preserving the volumetric structure, resulting in a loss of spatial coherence across slices. To overcome these limitations, we propose TomoGraphView, a novel framework that integrates omnidirectional volume slicing with spherical graph-based feature aggregation. We publicly share our accessible code base at http://github.com/compai-lab/2025-MedIA-kiechle and provide a user-friendly library for omnidirectional volume slicing at https://pypi.org/project/OmniSlicer.

Problem

• Automating 3D medical image classification to assist physicians with workload
• Overcoming limitations of 2D models applied to 3D volumes via slice decomposition
• Preserving volumetric spatial coherence in medical image analysis

Innovation

• Omnidirectional volume slicing captures spatial structures
• Graph-based aggregation preserves volumetric spatial coherence
• Leverages 2D foundation models for 3D medical analysis