Robust AI Evaluation through Maximal Lotteries

2026-02-24

AI Summary

This work addresses a fundamental limitation in conventional language model evaluation, which relies on Bradley-Terry ranking to compress heterogeneous human preferences into a total order—an approach that violates principles from social choice theory and fails to capture preference diversity. To overcome this, the authors propose a robust lottery mechanism that, for the first time, integrates robust optimization into the maximum lottery framework. By optimizing worst-case performance under adversarial preference perturbations without assuming strong structural constraints on preferences, the method supports a diverse set of winning models. Empirical validation on large-scale preference datasets demonstrates its effectiveness in providing more reliable win-rate guarantees and identifying top-performing models that exhibit consistent performance across distinct annotator groups, thereby advancing the development of AI evaluation systems aligned with the full spectrum of human preferences.

Technology Category

• Machine Learning: Learning Preferences or Rankings
• Humans and AI: Learning Human Values and Preferences
• Knowledge Representation and Reasoning: Preferences

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

The standard way to evaluate language models on subjective tasks is through pairwise comparisons: an annotator chooses the "better" of two responses to a prompt. Leaderboards aggregate these comparisons into a single Bradley-Terry (BT) ranking, forcing heterogeneous preferences into a total order and violating basic social-choice desiderata. In contrast, social choice theory provides an alternative approach called maximal lotteries, which aggregates pairwise preferences without imposing any assumptions on their structure. However, we show that maximal lotteries are highly sensitive to preference heterogeneity and can favor models that severely underperform on specific tasks or user subpopulations. We introduce robust lotteries that optimize worst-case performance under plausible shifts in the preference data. On large-scale preference datasets, robust lotteries provide more reliable win rate guarantees across the annotator distribution and recover a stable set of top-performing models. By moving from rankings to pluralistic sets of winners, robust lotteries offer a principled step toward an ecosystem of complementary AI systems that serve the full spectrum of human preferences.

Problem

• AI evaluation
• preference heterogeneity
• social choice
• language models
• leaderboard

Innovation

• robust lotteries
• maximal lotteries
• preference aggregation
• social choice theory
• language model evaluation