SAM-R1: Leveraging SAM for Reward Feedback in Multimodal Segmentation via Reinforcement Learning

2025-05-28

AI Summary

To address the bottleneck of relying on costly, fine-grained human annotations for reasoning supervision in multimodal image segmentation, this paper proposes an end-to-end reinforcement learning framework based on Proximal Policy Optimization (PPO), eliminating the need for explicit reasoning-step annotations. Methodologically, it integrates multimodal large language model (MLLM) fine-tuning, fine-grained reward modeling, and task-adaptive objective optimization. Its key contributions are twofold: (1) the first introduction of fine-grained segmentation modeling into MLLM reasoning training; and (2) the novel use of the Segment Anything Model (SAM) as a dynamic reward provider, generating high-quality, pixel-level dense rewards that enforce alignment between segmentation outputs and underlying reasoning logic. Evaluated on multiple benchmarks, the method achieves state-of-the-art performance using only 3K training samples—demonstrating significant improvements in segmentation accuracy, reasoning interpretability, and cross-task generalization.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Segmentation
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Leveraging multimodal large models for image segmentation has become a prominent research direction. However, existing approaches typically rely heavily on manually annotated datasets that include explicit reasoning processes, which are costly and time-consuming to produce. Recent advances suggest that reinforcement learning (RL) can endow large models with reasoning capabilities without requiring such reasoning-annotated data. In this paper, we propose SAM-R1, a novel framework that enables multimodal large models to perform fine-grained reasoning in image understanding tasks. Our approach is the first to incorporate fine-grained segmentation settings during the training of multimodal reasoning models. By integrating task-specific, fine-grained rewards with a tailored optimization objective, we further enhance the model's reasoning and segmentation alignment. We also leverage the Segment Anything Model (SAM) as a strong and flexible reward provider to guide the learning process. With only 3k training samples, SAM-R1 achieves strong performance across multiple benchmarks, demonstrating the effectiveness of reinforcement learning in equipping multimodal models with segmentation-oriented reasoning capabilities.

Problem

• Reducing reliance on costly manual reasoning-annotated datasets
• Enabling fine-grained reasoning in multimodal segmentation tasks
• Integrating SAM for efficient reward-guided reinforcement learning

Innovation

• Uses reinforcement learning for multimodal segmentation
• Integrates fine-grained rewards with optimization objective
• Leverages SAM as flexible reward provider