Beyond Accuracy: How Humans Evaluate Legally Correct but Socially Controversial Legal Advice from Machines

2026-07-06

AI Summary

This study investigates how the public evaluates legally sound yet socially contentious legal advice, focusing on how the source of the advice (AI versus human lawyers) and the inclusion of legal reasoning influence perceived reasonableness. Through a preregistered survey experiment involving 3,348 adults in mainland China, complemented by mediation analysis and qualitative content analysis, the research finds that attributing advice to AI exerts no significant net effect on perceived reasonableness—AI enhances perceived objectivity but diminishes perceived contextual sensitivity. In contrast, providing explicit legal reasoning significantly increases perceived reasonableness, primarily by bolstering perceptions of objectivity. These findings challenge conventional notions of “algorithm aversion,” revealing that public acceptance of AI-generated legal advice stems from a nuanced trade-off between objectivity and contextual awareness rather than an inherent bias against automation.

Abstract

AI systems are increasingly used to provide legal advice, raising questions about whether laypeople accept guidance from algorithms--especially when that advice is legally correct but socially controversial. We report a preregistered survey experiment with 3,348 adults in mainland China examining how people evaluate identical legal advice when it is attributed either to an AI system or to a human lawyer, and when it is accompanied by reasoning or not. Contrary to expectations of algorithm aversion, attribution to an AI system has no net effect on perceived reasonableness. However, mediation analyses reveal opposing psychological pathways underlying this null result. AI-attributed advice is perceived as more objective, which increases perceived reasonableness, but also as less comprehensive and less attentive to special circumstances, which decreases perceived reasonableness. By contrast, providing legal reasoning substantially increases perceived reasonableness regardless of source, largely by enhancing perceptions of objectivity. Qualitative responses corroborate this tension between objectivity and contextual sensitivity in evaluations of legal advice. Together, these findings suggest that public responses to AI legal advisors are shaped not by rigid attitudes toward automation, but by the balancing of competing normative expectations. The results have implications for theories of algorithm aversion and the design of AI recommendation systems in normatively salient domains.

Problem

• AI legal advice
• algorithm aversion
• perceived reasonableness
• social controversy
• normative expectations

Innovation

• algorithm aversion
• legal advice
• perceived reasonableness
• AI objectivity
• normative expectations