MedCore: Boundary-Preserving Medical Core Pruning for MedSAM

2026-05-13

AI Summary

This work addresses the challenge of deploying large medical segmentation foundation models, such as MedSAM, in clinical settings due to their substantial parameter count and the tendency of conventional compression methods to degrade boundary accuracy. To this end, the authors propose a structured pruning framework that preserves two critical types of structures within the image encoder: those essential for knowledge transfer from SAM and those exhibiting high leverage in boundary prediction. By introducing a boundary leverage principle that links boundary displacement to logit perturbations and spatial gradients, they devise a dual-intervention scoring mechanism and a boundary-aware Fisher information estimator. This approach enables effective pruning while maintaining both overall segmentation performance and boundary fidelity. On polyp segmentation, the method achieves 60.0% parameter reduction and 58.4% FLOPs savings, yielding a Dice score of 0.9549, Boundary F1 of 0.6388, and HD95 of 5.14; even with up to 86.6% parameter compression, boundary quality remains robust.

Abstract

Medical segmentation foundation models such as SAM and MedSAM provide strong prompt-driven segmentation, but their image encoders are still too large for many clinical settings. Compression is also risky in medicine because a model can keep high Dice while losing boundary fidelity. We propose MedCore, a structured pruning framework for MedSAM. The main idea is to preserve two kinds of structures: structures that became important during SAM-to-MedSAM adaptation, and structures that have high boundary leverage. We identify the first type by a dual-intervention score that compares zeroing a group with resetting it to its original SAM weight. We identify the second type by boundary-aware Fisher estimation. We also introduce a boundary leverage principle, which shows that compression-induced boundary displacement is controlled by logit perturbation on the boundary divided by the logit spatial gradient. This principle explains why boundary metrics can degrade even when Dice remains high. On polyp segmentation benchmarks, MedCore reduces parameters by 60.0% and FLOPs by 58.4% while achieving Dice 0.9549, Boundary F1 0.6388, and HD95 5.14 after recovery fine-tuning. It also reaches 86.6% parameter reduction and 90.4G FLOPs with strong boundary quality. Our analysis further shows that MedSAM lies in a head-fragile boundary regime: head-pruning steps have 2.887 times larger 95th-percentile boundary leverage than MLP-pruning steps, and this logit-level effect is consistent with BF1 and HD95 degradation. Our code is available at https://github.com/cenweizhang/MedCore.

Problem

• medical segmentation
• model compression
• boundary fidelity
• structured pruning
• foundation models

Innovation

• structured pruning
• boundary preservation
• medical foundation model
• boundary leverage
• MedSAM