Humans incorrectly reject confident accusatory AI judgments

2025-12-02

AI Summary

This study investigates human adoption mechanisms of high-confidence AI judgments in deception detection tasks. Addressing the critical question—“Do humans trust more accurate and confident AI deception assessments?”—we conducted a controlled experiment systematically manipulating AI model accuracy (high vs. low) and output confidence (high vs. low), while distinguishing accusatory judgments (asserting deception) from non-accusatory ones (asserting truthfulness). Results reveal that although participants overall prefer AI with higher accuracy, they significantly reject high-confidence accusatory judgments—even from highly accurate models—and human-AI collaboration degrades overall discrimination performance. This work provides the first empirical evidence of a “confidence–accusatoriness” interaction effect as a socio-cognitive bottleneck in human-AI decision integration. It offers foundational insights for designing trustworthy AI systems and robust human-AI collaboration frameworks, highlighting critical psychological constraints beyond technical reliability.

Technology Category

• Humans and AI: Learning Human Values and Preferences
• Cognitive Modeling & Cognitive Systems: Social Cognition And Interaction
• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness

Application Category

• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• User Modeling, Personalization and Recommendation: Accountability, Transparency, and Ethics for personalization

Abstract

Automated verbal deception detection using methods from Artificial Intelligence (AI) has been shown to outperform humans in disentangling lies from truths. Research suggests that transparency and interpretability of computational methods tend to increase human acceptance of using AI to support decisions. However, the extent to which humans accept AI judgments for deception detection remains unclear. We experimentally examined how an AI model's accuracy (i.e., its overall performance in deception detection) and confidence (i.e., the model's uncertainty in single-statements predictions) influence human adoption of the model's judgments. Participants (n=373) were presented with veracity judgments of an AI model with high or low overall accuracy and various degrees of prediction confidence. The results showed that humans followed predictions from a highly accurate model more than from a less accurate one. Interestingly, the more confident the model, the more people deviated from it, especially if the model predicted deception. We also found that human interaction with algorithmic predictions either worsened the machine's performance or was ineffective. While this human aversion to accept highly confident algorithmic predictions was partly explained by participants'tendency to overestimate humans'deception detection abilities, we also discuss how truth-default theory and the social costs of accusing someone of lying help explain the findings.

Problem

• Humans reject confident AI deception accusations
• AI accuracy and confidence affect human judgment adoption
• Human interaction worsens or is ineffective for AI performance

Innovation

• AI model accuracy increases human adoption
• High confidence AI predictions reduce human acceptance
• Human interaction worsens or is ineffective with algorithms