Think-with-Rubrics: From External Evaluator to Internal Reasoning Guidance

2026-05-08

AI Summary

This work addresses the limitation of existing approaches that treat rubrics merely as post-hoc evaluation tools, thereby failing to guide large language models during reasoning. The authors propose a novel “Think-with-Rubrics” paradigm, in which rubrics are embedded as intrinsic reasoning guides: the model first generates a rubric and then produces a response, with a specially trained rubric verifier jointly supervising the consistency between the two. This approach marks the first shift of rubrics from external evaluative mechanisms to internal reasoning scaffolds. By integrating both self-generated and gold-standard rubrics, the method achieves an average improvement of 3.87 points over baselines that rely solely on gold-rubric-based rewards across multiple benchmarks, demonstrating the synergistic enhancement of rubric quality and response alignment.

Abstract

Rubrics have been extensively utilized for evaluating unverifiable, open-ended tasks, with recent research incorporating them into reward systems for reinforcement learning. However, existing frameworks typically treat rubrics only as external evaluator disjointed from the policy's primary reasoning trace. Such design confines rubrics to post-hoc measurement, leaving them unable to actively guide the model's generation process. In this work, we introduce Think-with-Rubrics, a novel paradigm for instruction following tasks. Think-with-Rubrics integrates rubric generation into the reasoning context, transforming the rubric from an independent artifact into an internal guidance of LLM's generation. During training, LLM sequentially generates a rubric followed by a response, while a trained rubric verifier provides joint supervision by evaluating the consistency between the answer and the self-generated / golden rubrics. Experiments across multiple benchmarks demonstrate that Think-with-Rubrics consistently outperforms the Rubric-as-Reward baseline supervised by golden rubrics by an average of 3.87 points. We have also discussed the mechanism by which Think-with-Rubrics enhances model performance. Experimental results demonstrate that supervision from golden rubrics and self-generated rubrics enhances the performance of Think-with-Rubrics by improving the quality of self-generated rubrics and increasing the internal consistency of responses respectively.

Problem

• rubrics
• reasoning guidance
• instruction following
• large language models
• evaluation

Innovation

• Think-with-Rubrics
• rubric-guided reasoning
• internal guidance
• self-generated rubrics
• joint supervision