SeGDeP: Semantic- and Geometric-Aware Decoupled Prompts for Reasoning Segmentation

2026-09-08

AI Summary

本文提出SeGDeP方法，通过语义提示分支和几何投影路径解耦合地解决推理分割中目标识别与定位问题。

Abstract

Reasoning segmentation converts an implicit linguistic conclusion into a precise mask, requiring both semantic identification and spatial grounding. Existing MLLM-segmenter interfaces either use a special trigger or compress both signals into one context, although they receive different supervision and fail differently. This coupling obscures whether a failure arises from target interpretation or from localization. We present SeGDeP, an explicit what-where interface. A semantic prompt branch and an independent geometric projection path transform resolved MLLM states into semantic features and a DETR-predicted box, which jointly condition a SAM 3 mask decoder. Training first aligns this executable interface, then uses group reward-decoupled policy optimization (GDPO) to balance format, box-IoU, and mask-IoU feedback. SeGDeP-4B reaches 82.7 average cIoU over eight RefCOCO-family splits and 66.0/59.6 gIoU on ReasonSeg val/test while adapting only 0.38% of Qwen3-VL parameters through LoRA. Controlled stage-wise ablations, gradient diagnostics, and prompt interventions further show that the two paths develop complementary semantic and geometric specialization rather than duplicating the same evidence.

Problem

• reasoning segmentation
• semantic identification
• spatial grounding

Innovation

• Semantic-Geometric Decoupling
• DETR-predicted Box
• SAM 3 Mask Decoder
• Group Reward-Decoupled Policy Optimization (GDPO)
• LoRA