Same Words, Different Judgments: Modality Effects on Preference Alignment

2026-02-26

AI Summary

This study addresses the underexplored domain of preference alignment in speech modality, where annotation protocols remain ill-defined compared to the well-studied text modality. Through a cross-modal controlled experiment, the authors systematically compare human preference judgments for semantically identical content presented as either text or audio, employing intraclass correlation coefficient (ICC) to quantify annotation reliability. The work reveals, for the first time, that preference criteria differ significantly between modalities—exhibiting near-random cross-modal consistency—yet audio annotations achieve reliability comparable to text (ICC ≈ 0.80, requiring approximately nine annotators). Moreover, audio preferences demonstrate stronger user orientation and reduced length bias. The study further shows that synthetic scores can effectively predict human agreement, suggesting their potential to substitute manual annotation in preference modeling.

Technology Category

• Knowledge Representation and Reasoning: Preferences
• Machine Learning: Learning Preferences or Rankings
• Humans and AI: Learning Human Values and Preferences

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

Preference-based reinforcement learning (PbRL) is the dominant framework for aligning AI systems to human preferences, but its application to speech remains underexplored. We present a controlled cross-modal study of human and synthetic preference annotations, comparing text and audio evaluations of identical semantic content across 100 prompts. Audio preferences prove as reliable as text, with inter-rater agreement reaching good levels (ICC(2,k) $\approx$ .80) at $\sim$9 raters -- the first ICC-based reliability characterization in the preference annotation literature for either modality. However, modality reshapes how people judge: audio raters exhibit narrower decision thresholds, reduced length bias, and more user-oriented evaluation criteria, with near-chance cross-modality agreement. Synthetic ratings further align with human judgments and predict inter-rater agreement, supporting their use both for triaging ambiguous pairs and as full replacements for human annotations.

Problem

• preference alignment
• modality effects
• speech
• text
• human preferences

Innovation

• preference alignment
• modality effects
• speech preference
• reliability analysis
• synthetic annotations