Problem
Research questions and friction points this paper is trying to address.
AI Perception
Model Updates
User Behavior
What Lies Beneath? Exploring the Impact of Underlying AI Model Updates in AI-Infused Systems
🤖 AI Summary
This study investigates how implicit model updates in AI systems affect users’ perception, cognition, and behavior—specifically in historical figure face recognition—where such updates often occur without user awareness or consent. Method: We employed a dual-method design: (1) an A/B online experiment and (2) a longitudinal field diary study, complemented by behavioral logging, qualitative interviews, and comparative analysis using a FaceNet variant. Contribution/Results: We first empirically identify a significant perceptual blind spot: 62% of users failed to detect minor model updates. Unaware of these changes, users developed biased “folk theories” about system behavior, leading to diminished trust, misaligned interaction strategies, and reduced task completion rates. Based on these findings, we propose a transparency framework for human-AI co-evolution, supporting sustainable, explainable, and trustworthy AI updates. The framework advances theoretical understanding and provides actionable design principles for long-term AI system usability and update governance.
📝 Abstract
AI models are constantly evolving, with new versions released frequently. This raises a key question: how should AI-infused systems integrate updates when the downstream impact on user experience and performance is unclear? Human-AI interaction guidelines encourage notifying users about (changes in) model capabilities, ideally supported by thorough benchmarking. Yet, as AI models integrate into domain-specific workflows, exhaustive benchmarking can become impractical or expensive, often leading to invisible or minimally communicated updates. In this work, we explore the impact of such updates through two complementary studies on facial recognition for historical person identification. First, we conducted an online experiment to understand how users distinguish between models, followed by a diary study examining user perceptions in a real-world deployment. Our findings reveal how model changes impact human-AI performance, downstream user behavior, and the folk theories they develop. Based on these insights, we discuss implications for updating models in AI-infused systems.
Innovation
Methods, ideas, or system contributions that make the work stand out.
AI Model Updates
User Perception
Effective Notification Strategies
🔎 Similar Papers
No similar papers found.
