Learning from an Unknown DGP: Experimental Evidence on Belief Updating with AI Recommendations

2026-07-11

AI Summary

This study investigates how humans update their beliefs when confronted with qualitative AI recommendations derived from an unknown data-generating process. Drawing on a controlled behavioral experiment comprising 60,252 prior–posterior belief pairs, the authors systematically compare the performance of three prominent belief-updating models through integrated modeling and statistical analysis. The findings reveal asymmetric belief updating under extreme priors and significantly attenuated adjustments under moderate priors. Building on these patterns, the study articulates four testable properties of belief updating and identifies three archetypal behavioral profiles. Model applicability is validated at both individual and aggregate levels, offering crucial empirical foundations for AI-assisted decision-making.

Abstract

We use a controlled experiment to study how beliefs are updated after receiving qualitative information (AI recommendations) from an unknown data-generating process (DGP). Across 60,252 pairs of prior and posterior beliefs, we document three behavioral patterns: updates close to zero when recommendations confirm extreme priors, larger updates when recommendations contradict extreme priors, and smaller updates for intermediate priors. These three behavioral patterns suggest four testable properties of belief updating, which we assess at the aggregate and individual levels. Finally, we examine how well updates are captured by three models of belief updating.

Problem

• belief updating
• AI recommendations
• unknown DGP
• behavioral patterns
• prior and posterior beliefs

Innovation

• belief updating
• AI recommendations
• unknown DGP
• behavioral patterns
• controlled experiment