PointRL: Learning Point-Level Vision-Language Grounding from Verifiable Annotation Evidence

2026-08-25

AI Summary

本文提出PointRL框架，通过强化学习和异构标注证据学习点级视觉-语言对齐，解决多实例指令中的目标覆盖、计数一致性和重复抑制问题。

Abstract

Vision-language models (VLMs) increasingly rely on point coordinates as a compact and executable interface for visual grounding in GUI interaction, robotic manipulation, and interactive visual systems. However, learning reliable pointing behavior remains difficult because the supervision space is inherently non-unique: many coordinates may be valid within the same target region, while multi-instance instructions require target coverage, count consistency, and duplicate suppression. This work presents PointRL, a verifiable reinforcement learning framework that learns point-level grounding from existing heterogeneous annotation evidence. PointRL converts bounding boxes, masks, and instance labels into pointing instructions, while retaining their target supports, instance membership, and set constraints as hidden verifier evidence, i.e., annotations kept outside the prompt and used by a deterministic checker to score predictions. The proposed reward evaluates parseability, point validity, instance coverage, cardinality consistency, and redundant or missing predictions. On PointArena, PointRL improves the overall accuracy of Qwen3.5-4B from 56.11% to 65.58%. Further evaluations on RoboSpatial, BLINK, and Ref-Adv show same-backbone gains on the evaluated external benchmarks, suggesting that verifiable point-level feedback may benefit spatial grounding in these settings.

Problem

• point-level grounding
• non-unique supervision space
• multi-instance instructions
• target coverage
• count consistency

Innovation

• verifiable reinforcement learning
• point-level grounding
• annotation evidence
• deterministic checker
• reward mechanism