SMAFormer: Synergistic Multi-Attention Transformer for Medical Image Segmentation

2024-08-31

IEEE International Conference on Bioinformatics and Biomedicine

AI Summary

To address the low segmentation accuracy of irregularly shaped and minute tumors in medical images, this paper proposes a Synergistic Multi-Attention Transformer (SMA-Transformer). The architecture innovatively integrates pixel-wise, channel-wise, and spatial-wise attention mechanisms, coupled with a feature fusion modulator to jointly model local details and global context—thereby mitigating information loss during attention transformation and feature recalibration. Its modular design incorporates residual connections to enhance gradient flow and feature reuse. Evaluated on multi-organ, liver tumor, and bladder tumor segmentation tasks, the method achieves state-of-the-art (SOTA) performance, particularly for small-target segmentation. It delivers significant improvements in key metrics: Dice coefficient and 95th-percentile Hausdorff Distance (HD95), demonstrating superior accuracy and boundary localization.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Large Multimodal Models (LMMs)
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs

Abstract

In medical image segmentation, specialized computer vision techniques, notably transformers grounded in attention mechanisms and residual networks employing skip connections, have been instrumental in advancing performance. Nonetheless, previous models often falter when segmenting small, irregularly shaped tumors. To this end, we introduce SMAFormer, an efficient, Transformer-based architecture that fuses multiple attention mechanisms for enhanced segmentation of small tumors and organs. SMAFormer can capture both local and global features for medical image segmentation. The architecture comprises two pivotal components. First, a Synergistic Multi-Attention (SMA) Transformer block is proposed, which has the benefits of Pixel Attention, Channel Attention, and Spatial Attention for feature enrichment. Second, addressing the challenge of information loss incurred during attention mechanism transitions and feature fusion, we design a Feature Fusion Modulator. This module bolsters the integration between the channel and spatial attention by mitigating reshaping-induced information attrition. To evaluate our method, we conduct extensive experiments on various medical image segmentation tasks, including multi-organ, liver tumor, and bladder tumor segmentation, achieving state-of-the-art results. Code and models are available at: https://github.com/lzeeorno/SMAFormer.

Problem

• Segmenting small irregularly shaped tumors in medical images
• Integrating multiple attention mechanisms for feature enrichment
• Mitigating information loss during attention transitions and fusion

Innovation

• Fuses multiple attention mechanisms for small tumors
• Uses Synergistic Multi-Attention Transformer block
• Designs Feature Fusion Modulator to prevent information loss