Multi-Task Reinforcement Learning for Enhanced Multimodal LLM-as-a-Judge

2026-03-12

AI Summary

Existing MLLM-as-a-Judge approaches are typically optimized for individual tasks, resulting in limited generalization capabilities. This work proposes the first integration of multi-task reinforcement learning into this paradigm, jointly training a multimodal large language model across diverse tasks to simultaneously enhance judgment consistency and alignment with human preferences. The proposed method significantly outperforms strong baselines on both in-distribution and out-of-distribution tasks, achieving state-of-the-art performance across multiple evaluation metrics. Moreover, it demonstrates remarkable generalization ability and produces assessment outcomes that closely align with human judgments.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Multiagent Systems: Multiagent Learning

Application Category

• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Multimodal Large Language Models (MLLMs) have been widely adopted as MLLM-as-a-Judges due to their strong alignment with human judgment across various visual tasks. However, most existing judge models are optimized for single-task scenarios and struggle to generalize to diverse contexts, which is a critical requirement for reliable evaluation. To address this limitation, we propose Multi-Task Reinforcement Learning for MLLM-as-a-Judge (MT-RL-Judge), a framework that jointly optimizes the judge model across multiple tasks, leveraging the generalization capabilities of RL. Experimental results against several strong baselines demonstrate that MT-RL-Judge outperforms strong baselines in both judgment consistency and correlation with human preferences. Furthermore, our approach exhibits robust generalization on out-of-distribution tasks, further validating its effectiveness.

Problem

• Multimodal Large Language Models
• MLLM-as-a-Judge
• multi-task generalization
• evaluation reliability
• out-of-distribution generalization

Innovation

• Multi-Task Reinforcement Learning
• MLLM-as-a-Judge
• Multimodal Large Language Models
• Generalization
• Human Preference Alignment