ReliableEval: A Recipe for Stochastic LLM Evaluation via Method of Moments

2025-05-28

AI Summary

Existing LLM evaluation relies on single-prompt measurements, yet LLMs exhibit high sensitivity to minor prompt perturbations, yielding unreliable and non-reproducible performance estimates. Method: We propose the first formal definition of *reliable evaluation* and introduce a gradient-free, task-agnostic randomized evaluation framework. It operates over a semantic-preserving prompt perturbation space, quantifying prompt sensitivity via moment estimation (first- and second-order) and statistical confidence interval analysis, while adaptively determining optimal sampling size. Contribution/Results: The framework is compatible with arbitrary LLMs, tasks, and evaluation metrics. Experiments across five state-of-the-art models—including GPT-4o and Claude-3.7-Sonnet—demonstrate that single-prompt evaluations frequently incur biases exceeding 15%, whereas our method substantially improves assessment stability and reproducibility without requiring model access or fine-tuning.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Safety and Robustness
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

LLMs are highly sensitive to prompt phrasing, yet standard benchmarks typically report performance using a single prompt, raising concerns about the reliability of such evaluations. In this work, we argue for a stochastic method of moments evaluation over the space of meaning-preserving prompt perturbations. We introduce a formal definition of reliable evaluation that accounts for prompt sensitivity, and suggest ReliableEval - a method for estimating the number of prompt resamplings needed to obtain meaningful results. Using our framework, we stochastically evaluate five frontier LLMs and find that even top-performing models like GPT-4o and Claude-3.7-Sonnet exhibit substantial prompt sensitivity. Our approach is model-, task-, and metric-agnostic, offering a recipe for meaningful and robust LLM evaluation.

Problem

• Addressing LLM sensitivity to prompt phrasing in evaluations
• Proposing stochastic evaluation over meaning-preserving prompt variations
• Estimating required prompt resamplings for reliable LLM assessment

Innovation

• Stochastic evaluation over prompt perturbations
• Estimates required prompt resamplings for reliability
• Model-task-metric agnostic robust evaluation framework