🤖 AI Summary
This study addresses the lack of systematic evaluation regarding the reliability and alignment with human judgment of large language models (LLMs) when deployed as automated evaluators. The authors construct a human-annotated gold-standard dataset spanning eight distinct tasks and conduct the first large-scale empirical analysis of 37 open- and closed-source conversational LLMs under five裁判 prompting strategies, a two-stage judging mechanism, and task-specific fine-tuning. Results demonstrate that GPT-4o, open-source models with at least 32 billion parameters, and Qwen2.5-14B achieve high agreement with human judgments when paired with appropriate prompts, thereby validating the feasibility of using LLMs as reliable automated evaluators. The findings offer empirical guidance for prompt design, model selection, and architectural optimization in automated assessment systems.
📝 Abstract
A Large Language Model (LLM) as judge evaluates the quality of victim Machine Learning (ML) models, specifically LLMs, by analyzing their outputs. An LLM as judge is the combination of one model and one specifically engineered judge prompt that contains the criteria for the analysis. The resulting automation of the analysis scales up the complex evaluation of the victim models' free-form text outputs by faster and more consistent judgments compared to human reviewers. Thus, quality and security assessments of LLMs can cover a wide range of the victim models' use cases. Being a comparably new technique, LLMs as judges lack a thorough investigation for their reliability and agreement to human judgment.
Our work evaluates the applicability of LLMs as automated quality assessors of victim LLMs. We test the efficacy of 37 differently sized conversational LLMs in combination with 5 different judge prompts, the concept of a second-level judge, and 5 models fine-tuned for the task as assessors. As assessment objective, we curate datasets for eight different categories of judgment tasks and the corresponding ground-truth labels based on human assessments. Our empirical results show a high correlation of LLMs as judges with human assessments, when combined with a suitable prompt, in particular for GPT-4o, several open-source models with $\geqslant$ 32B parameters, and a few smaller models like Qwen2.5 14B.