TopoMamba: Topology-Aware Scanning and Fusion for Segmenting Heterogeneous Medical Visual Media

2026-04-28

AI Summary

Existing visual state space models struggle to accurately segment oblique or curved anatomical structures due to axial scanning bias and inefficient multi-branch fusion. To address this, this work proposes a topology-aware scanning and fusion framework that integrates diagonal, anti-diagonal, and cross-scanning strategies. A novel ScanCache mechanism is introduced to enhance computational efficiency, alongside a lightweight gating fusion module grounded in the Hilbert-Schmidt Independence Criterion (HSIC). The proposed method significantly outperforms CNN-, Transformer-, and SSM-based baselines on multiple benchmarks—including Synapse CT, ISIC 2017, and CVC-ClinicDB—demonstrating particularly strong performance in segmenting complex, slender structures such as the pancreas and gallbladder. Moreover, the framework supports efficient deployment across dynamic input resolutions.

Abstract

Visual state-space models (SSMs) have shown strong potential for medical image segmentation, yet their effectiveness is often limited by two practical issues: axis-biased scan ordering weakens the modeling of oblique and curved structures, and naive multi-branch fusion tends to amplify redundant responses. We present TopoMamba, a topology-aware scan-and-fuse framework for segmenting heterogeneous medical visual media. The method combines a diagonal/anti-diagonal TopoA-Scan branch with the standard Cross-Scan branch to provide complementary structural priors, and introduces ScanCache, a device-aware caching mechanism that amortizes explicit scan-index construction across recurring resolutions. To fuse heterogeneous scan features efficiently, we further propose a lightweight HSIC Gate that regulates branch interaction using a dependence-aware scalar gating rule. We also instantiate a volumetric TopoMamba-3D for practical 3D clinical segmentation. Experiments on Synapse CT, ISIC 2017 dermoscopy, and CVC-ClinicDB endoscopy show that TopoMamba consistently improves segmentation quality over strong CNN, Transformer, and SSM baselines, with particularly clear gains on thin or curved targets such as the pancreas and gallbladder, while maintaining favorable deployment efficiency under dynamic input resolutions. These results suggest that topology-aware scan ordering and lightweight dependence-aware fusion form an effective and practical design for medical multimedia segmentation. The code will be made publicly available.

Problem

• medical image segmentation
• scan ordering
• multi-branch fusion
• curved structures
• visual state-space models

Innovation

• Topology-aware scanning
• Visual state-space models
• HSIC Gate
• ScanCache
• Medical image segmentation