MultEval: Supporting Collaborative Alignment for LLM-as-a-Judge Evaluation Criteria

2026-04-29

AI Summary

Current evaluation benchmarks for large language models (LLMs) are typically established by a single entity, failing to reflect the diverse values of multiple stakeholders and lacking support for collaborative development and iterative refinement. This work introduces MultEval, the first system to integrate consensus-building mechanisms into the LLM evaluation standard formulation process. MultEval combines consensus theory, an interactive collaborative interface, example anchoring, and versioned standard management to enable disagreement diagnosis, transparent justification of judgments, and traceable evolution of evaluation criteria. In case studies with domain experts, MultEval effectively facilitated multi-stakeholder consensus and clearly captured the dynamic evolution of evaluation standards through collaborative negotiation.

Abstract

LLM-as-a-judge approaches have emerged as a scalable solution for evaluating model behaviors, yet they rely on evaluation criteria often created by a single individual, embedding that person's assumptions, priorities, and interpretive lens. In practice, defining such criteria is a collaborative and contested process involving multiple stakeholders with different values, interpretations, and priorities; an aspect largely unsupported by existing tools. To examine this problem in depth, we present a formative study examining how stakeholders collaboratively create, negotiate, and refine evaluation criteria for LLM-as-a-judge systems. Our findings reveal challenges in human oversight, including difficulties in establishing shared understanding, aligning values across stakeholders with different expertise and priorities, and translating nuanced human judgments into criteria that are interpretable and actionable for LLM judges. Based on these insights, we developed MultEval, a system that supports collaborative criteria by enabling multiple evaluators to surface and diagnose disagreements using consensus-building theory, iteratively revise criteria with attached examples and proposal history, and maintain transparency over how judgments are encoded into an automated evaluator. We further report a case study in which a team of domain experts used MultEval to collaboratively author criteria, illustrating how coordination and collaborative consensus-making shape criteria evolution.

Problem

• LLM-as-a-judge
• evaluation criteria
• collaborative alignment
• stakeholder values
• human oversight

Innovation

• collaborative alignment
• LLM-as-a-judge
• evaluation criteria
• consensus building
• human-AI evaluation