HWA-UNETR: Hierarchical Window Aggregate UNETR for 3D Multimodal Gastric Lesion Segmentation

2025-05-15

AI Summary

Gastric cancer segmentation from multimodal MRI faces dual challenges: inter-modal anatomical misalignment and severe scarcity of annotated data, limiting the generalizability and accuracy of existing transfer-learning-based approaches. To address these, we introduce GCM 2025—the first publicly available multimodal gastric MRI dataset comprising 500 fully paired T1-, T2-, and FLAIR-weighted scans with expert annotations. We propose a Hybrid Warping Alignment (HWA) module for cross-modal dynamic registration and a tri-directional directional fusion Mamba mechanism integrated with a 3D Transformer to jointly optimize long-range spatial modeling and multimodal feature synergy. Evaluated on GCM 2025 and BraTS 2021, our method achieves a 1.68% Dice score improvement over state-of-the-art baselines, demonstrating superior segmentation accuracy and robustness. Both the dataset and source code are fully open-sourced.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Multimodal medical image segmentation faces significant challenges in the context of gastric cancer lesion analysis. This clinical context is defined by the scarcity of independent multimodal datasets and the imperative to amalgamate inherently misaligned modalities. As a result, algorithms are constrained to train on approximate data and depend on application migration, leading to substantial resource expenditure and a potential decline in analysis accuracy. To address those challenges, we have made two major contributions: First, we publicly disseminate the GCM 2025 dataset, which serves as the first large-scale, open-source collection of gastric cancer multimodal MRI scans, featuring professionally annotated FS-T2W, CE-T1W, and ADC images from 500 patients. Second, we introduce HWA-UNETR, a novel 3D segmentation framework that employs an original HWA block with learnable window aggregation layers to establish dynamic feature correspondences between different modalities' anatomical structures, and leverages the innovative tri-orientated fusion mamba mechanism for context modeling and capturing long-range spatial dependencies. Extensive experiments on our GCM 2025 dataset and the publicly BraTS 2021 dataset validate the performance of our framework, demonstrating that the new approach surpasses existing methods by up to 1.68% in the Dice score while maintaining solid robustness. The dataset and code are public via https://github.com/JeMing-creater/HWA-UNETR.

Problem

• Addresses scarcity of multimodal datasets for gastric cancer analysis
• Solves misalignment issues in combining different medical imaging modalities
• Improves accuracy and reduces resource use in lesion segmentation

Innovation

• HWA block with learnable window aggregation layers
• Tri-orientated fusion mamba mechanism
• Dynamic feature correspondences between modalities