Robust Tumor Segmentation with Hyperspectral Imaging and Graph Neural Networks

2023-11-20

arXiv.org

AI Summary

Accurate intraoperative tumor boundary delineation remains a critical surgical challenge. Conventional hyperspectral imaging (HSI)-based machine learning approaches typically perform pixel-wise or superpixel-wise classification, neglecting spatial contextual information—leading to non-robust segmentation and jagged tumor boundaries. To address this, we propose a hybrid CNN-GNN framework: a convolutional neural network (CNN) extracts local spectral-spatial features, while a graph neural network (GNN) models topological relationships among irregular tissue regions. Crucially, we introduce, for the first time, an image-quality-aware weighted loss function to enable robust patch-level segmentation. By transcending the limitations of isolated superpixel modeling, our method achieves significant improvements over context-agnostic baselines on 51 ex vivo clinical HSI images. It demonstrates superior cross-patient generalizability and boundary precision, establishing a novel paradigm for real-time intraoperative navigation.

Abstract

Segmenting the boundary between tumor and healthy tissue during surgical cancer resection poses a significant challenge. In recent years, Hyperspectral Imaging (HSI) combined with Machine Learning (ML) has emerged as a promising solution. However, due to the extensive information contained within the spectral domain, most ML approaches primarily classify individual HSI (super-)pixels, or tiles, without taking into account their spatial context. In this paper, we propose an improved methodology that leverages the spatial context of tiles for more robust and smoother segmentation. To address the irregular shapes of tiles, we utilize Graph Neural Networks (GNNs) to propagate context information across neighboring regions. The features for each tile within the graph are extracted using a Convolutional Neural Network (CNN), which is trained simultaneously with the subsequent GNN. Moreover, we incorporate local image quality metrics into the loss function to enhance the training procedure's robustness against low-quality regions in the training images. We demonstrate the superiority of our proposed method using a clinical ex vivo dataset consisting of 51 HSI images from 30 patients. Despite the limited dataset, the GNN-based model significantly outperforms context-agnostic approaches, accurately distinguishing between healthy and tumor tissues, even in images from previously unseen patients. Furthermore, we show that our carefully designed loss function, accounting for local image quality, results in additional improvements. Our findings demonstrate that context-aware GNN algorithms can robustly find tumor demarcations on HSI images, ultimately contributing to better surgery success and patient outcome.

Problem

• Enhancing tumor segmentation accuracy
• Integrating spatial context in HSI analysis
• Improving robustness with GNN and CNN

Innovation

• Uses Graph Neural Networks
• Integrates spatial context
• Incorporates local quality metrics