Problem
Research questions and friction points this paper is trying to address.
LLM-as-Judge
trustworthiness
truth classification
separability
When Trust Meets Truth: Trust-Truth Separability in LLM-as-Judge
🤖 AI Summary
研究测试了大语言模型作为裁判时信任评分与真实性判断之间的分离性，通过改变信息来源进行压力测试，发现两者并不完全独立。
📝 Abstract
LLM-as-Judge systems can produce multi-dimensional evaluations, such as trustworthiness, reliability, and factuality, and these outputs are often interpreted as independent evidence. We test this assumption for a common pair of judgments: trust scoring and binary truth classification. On correctness-controlled QA, LLM judges align trust scores with truth verdicts more tightly than human behavioral reference, suggesting weaker separations between trust and truth judgment. We then apply stress tests by changing only source cues of identical QA between Human and AI. Source attribution shifts not only trust scores but also truth verdicts and logit-derived correct-side probabilities. Results show that current LLM-as-Judge protocols should not treat trust scores as independent evidence for truth judgments.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Trust-Truth Separability
LLM-as-Judge
Correctness-Controlled QA
🔎 Similar Papers
No similar papers found.
