More Isn't Always Better: Balancing Decision Accuracy and Conformity Pressures in Multi-AI Advice

2026-03-23

AI Summary

This study investigates how multiple AI recommendations influence human decision-making accuracy and reliance, particularly in light of potential conformity pressure. Through three behavioral experiments, the authors systematically manipulate group size, internal consensus among AI agents, and the degree of anthropomorphic presentation. Findings reveal that small AI groups (e.g., three agents) significantly outperform both single-AI and large-group configurations. Introducing moderate disagreement—such as a single dissenting AI—effectively mitigates conformity pressure without compromising decision quality. While anthropomorphic presentation enhances perceived usefulness of AI advice, it does not exacerbate overreliance or conformity risks. These results delineate boundary conditions for effectively deploying multi-AI advisory systems and offer theoretical foundations for designing hybrid decision-making frameworks that balance accuracy with human autonomy.

Technology Category

• Humans and AI: Planning and Decision Support for Human-Machine Teams
• Multiagent Systems: Agent/AI Theories and Architectures
• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Economics and fairness of platforms and recommendation systems

Abstract

Just as people improve decision-making by consulting diverse human advisors, they can now also consult with multiple AI systems. Prior work on group decision-making shows that advice aggregation creates pressure to conform, leading to overreliance. However, the conditions under which multi-AI consultation improves or undermines human decision-making remain unclear. We conducted experiments with three tasks in which participants received advice from panels of AIs. We varied panel size, within-panel consensus, and the human-likeness of presentation. Accuracy improved for small panels relative to a single AI; larger panels yielded no gains. The level of within-panel consensus affected participants' reliance on AI advice: High consensus fostered overreliance; a single dissent reduced pressure to conform; wide disagreement created confusion and undermined appropriate reliance. Human-like presentations increased perceived usefulness and agency in certain tasks, without raising conformity pressure. These findings yield design implications for presenting multi-AI advice that preserve accuracy while mitigating conformity.

Problem

• multi-AI advice
• decision accuracy
• conformity pressure
• overreliance
• advice aggregation

Innovation

• multi-AI advice
• conformity pressure
• advice aggregation
• human-AI collaboration
• decision accuracy