TopoLoRA-SAM: Topology-Aware Parameter-Efficient Adaptation of Foundation Segmenters for Thin-Structure and Cross-Domain Binary Semantic Segmentation
This work addresses the challenges faced by foundational segmentation models like SAM in fine-structured and cross-modal binary semantic segmentation tasks, where full fine-tuning is computationally expensive and prone to catastrophic forgetting. To overcome these limitations, we propose TopoLoRA-SAM—the first framework integrating topological awareness with parameter-efficient fine-tuning. Our approach freezes the ViT encoder and injects low-rank adaptation (LoRA) modules alongside lightweight spatial convolutional adapters, optionally augmented with differentiable clDice-based topological supervision. Training only 5.2% of the model parameters (approximately 4.9M), TopoLoRA-SAM achieves state-of-the-art average Dice scores across five benchmark datasets, with particularly significant improvements in segmentation accuracy and robustness on CHASE_DB1, outperforming fully fine-tuned task-specific models.