AdaJudge: Adaptive Multi-Perspective Judging for Reward Modeling

2026-01-13

AI Summary

This work addresses the limitations of existing reward models, which rely on static pooling strategies that introduce inductive biases misaligned with task-specific preferences and constrain representational capacity. To overcome these issues, we propose AdaJudge, a novel framework featuring a gated refinement module that maps backbone representations into a discrimination-oriented space, coupled with an adaptive multi-view pooling mechanism that dynamically fuses multi-dimensional evidence to produce reward scores. By jointly optimizing representation learning and aggregation strategies, AdaJudge enables adaptive reward modeling tailored to discriminative tasks. Extensive evaluations on RM-Bench and JudgeBench demonstrate that AdaJudge significantly outperforms current reward models and static pooling baselines, confirming its effectiveness and state-of-the-art performance.

Technology Category

• Multiagent Systems: Adversarial Agents
• Cognitive Modeling & Cognitive Systems: Adaptive Behavior
• Game Theory and Economic Paradigms: Adversarial Learning

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

Reward modeling is essential for aligning large language models with human preferences, yet predominant architectures rely on a static pooling strategy to condense sequences into scalar scores. This paradigm, however, suffers from two key limitations: a static inductive bias that misaligns with task-dependent preference signals, and a representational mismatch, as the backbone is optimized for generation rather than fine-grained discrimination. To address this, we propose AdaJudge, a unified framework that jointly adapts representation and aggregation. AdaJudge first refines backbone representations into a discrimination-oriented space via gated refinement blocks. It then replaces the static readout with an adaptive multi-view pooling module that dynamically routes and combines evidence. Extensive experiments on RM-Bench and JudgeBench show that AdaJudge outperforms strong off-the-shelf reward models and traditional pooling baselines.

Problem

• reward modeling
• static pooling
• inductive bias
• representational mismatch
• preference alignment

Innovation

• adaptive pooling
• discrimination-oriented representation
• reward modeling
• multi-perspective judging
• gated refinement