HPR-SAM: Hierarchical Probabilistic Representation Learning for Prompt-free SAM-based Medical Image Segmentation

2026-07-07

AI Summary

Existing prompt-free SAM-based medical image segmentation methods rely on deterministic representations, which struggle to simultaneously capture the global anatomical priors, internal structural diversity, and local reliability essential for accurate segmentation. To address this limitation, this work proposes the first hierarchical probabilistic representation learning framework that jointly models anatomical structures through Distributional Anatomical Representation (DAR), Multi-Group Anatomical Representation (MAR), and Local Reliability Representation (LRR). A Hierarchical Prediction Fusion (HPF) mechanism is introduced to integrate multi-scale information while remaining compatible with the original SAM decoder. The proposed method achieves state-of-the-art performance on the Synapse dataset and demonstrates superior results under few-shot settings on both the LA and PROMISE12 datasets, significantly surpassing the constraints of conventional deterministic representations.

Abstract

Prompt-free adaptation of the Segment Anything Model (SAM) has emerged as a promising paradigm for automatic medical image segmentation. Existing methods mainly focus on prompt generation, while overlooking that prompt quality is fundamentally constrained by the expressiveness of anatomical representations. However, deterministic prototypes or semantic tokens are insufficient to jointly capture global anatomical priors, intra-structure diversity, and local structural reliability. To address this limitation, we propose the Hierarchical Probabilistic Representation (HPR) framework, which learns complementary anatomical representations through Distributional Anatomical Representation (DAR), Multi-component Anatomical Representation (MAR), and Local Reliability Representation (LRR), and integrates their predictions via Hierarchical Prediction Fusion (HPF) while remaining compatible with the original SAM decoder. Experiments on the Synapse, LA, and PROMISE12 datasets demonstrate that HPR-SAM achieves state-of-the-art performance on Synapse and the best performance under few-shot settings on LA and PROMISE12, validating the effectiveness of the proposed hierarchical probabilistic representation learning framework for prompt-free medical image segmentation. Code is available at https://anonymous.4open.science/r/HPR-SAM-E4AF.

Problem

• prompt-free segmentation
• medical image segmentation
• anatomical representation
• Segment Anything Model
• representation expressiveness

Innovation

• Hierarchical Probabilistic Representation
• Prompt-free Segmentation
• Medical Image Segmentation
• Segment Anything Model
• Anatomical Representation Learning