RubricRM: Generative Reward Modeling via Dynamic Rubrics for Image Generation and Editing

2026-08-27

AI Summary

本文提出RubricRM，一种生成式奖励建模框架，通过动态评估标准解决图像生成和编辑中现有奖励模型适应性差的问题。

Abstract

Reward models play an essential role in aligning visual generative models, yet most existing visual reward models use a single scalar score or rely on fixed criteria that cannot adapt to different instructions. This limits both interpretability and task sensitivity, especially for text-to-image generation and instruction-based image editing, where different inputs require different evaluation dimensions. We propose RubricRM, a pairwise generative reward modeling framework that first produces an input-specific rubric with evaluation dimensions, weights, and scoring criteria, and then applies the rubric to score candidate images. We train dedicated RubricRM models for text-to-image generation and image editing using a two-stage training pipeline: supervised fine-tuning teaches the model the rubric-based scoring paradigm, while GRPO further improves scoring through fine-grained dimension-level rewards. Experiments on multiple generation and editing benchmarks show that RubricRM outperforms existing specialized reward models and remains competitive with strong proprietary MLLM judges despite using smaller backbones. Our models, data, and code are available at https://github.com/zijiankan/RubricRM.

Problem

• visual reward models
• interpretability
• task sensitivity
• text-to-image generation
• instruction-based image editing

Innovation

• Generative Reward Modeling
• Dynamic Rubrics
• Pairwise Framework
• Supervised Fine-Tuning
• Dimension-Level Rewards