SetPlanner: A Lightweight Plug-in Point-Set Planner for Frozen SAM

2026-09-15

AI Summary

为了解决自动提示难以处理细长、反光和部分遮挡工具的问题，本文提出SetPlanner，通过轻量级点集规划方法来改进Frozen SAM的自动分割能力。

Abstract

Segment Anything Models provide reusable priors, yet they require user prompts and cannot support fully automatic instrument segmentation. Automatic prompting is difficult for thin, articulated, reflective, and partly occluded tools, where several configurations can be valid. We formulate automatic prompting as lightweight point-set planning and isolate the point source under a frozen pathway. To this end, we present SetPlanner, a 1.52M-parameter plug-in point-set planner for frozen SAM. The plug-in preserves SAM's point-prompt interface and enables reuse across backbones. SetPlanner plans complete unordered K-point sets from geometry-aware targets with a permutation-aware conditional flow. SAM decodes eight candidates; their consensus readout yields a ground-truth-free prediction. Across three endoscopic datasets, SetPlanner wins all six transfer routes over a LoRA-adapted system. Under our frozen-pathway protocol, SetPlanner reaches 0.934 Dice on Kvasir-Instrument and recovers 96% of a 44.4-point localization gap, while candidate disagreement ranks low-Dice cases at AUROC 0.969.

Problem

• Segment Anything Models
• automatic prompting
• instrument segmentation

Innovation

• SetPlanner
• point-set planning
• frozen SAM
• geometry-aware targets
• permutation-aware conditional flow