Perceived System Predictability: Scale Development and Application

2026-07-06

AI Summary

This study addresses the lack of a clear definition and effective measurement of user-perceived system predictability (PSP) in human–computer interaction, which has hindered understanding of trust and reliance mechanisms. Grounded in uncertainty theory, it conceptualizes PSP for the first time as a multidimensional construct encompassing cognitive, stochastic, and effective predictability, clarifies its distinction from related constructs such as trust and understanding, and reveals its dissociation from prediction accuracy. Through expert review, cognitive interviews, and two controlled experiments involving shape and emotion classification tasks, the authors developed and validated a six-item scale, confirming both a unidimensional and a three-level hierarchical factor structure. Results demonstrate that PSP predicts users’ prediction accuracy; explanation styles influence PSP without affecting accuracy; and system randomness reduces accuracy but leaves PSP unchanged.

Abstract

How predictable users perceive an interactive system to be shapes how they interpret, trust, and rely on it, yet HCI lacks both a precise conceptualization and a validated instrument for this perception. We address this gap by introducing perceived system predictability (PSP) as a user-centered construct grounded in uncertainty theory, distinguishing epistemic, aleatory, and effective predictability. We contribute (i) a theoretical framework that situates PSP relative to adjacent constructs such as trust and understanding, (ii) a 6-item PSP scale, derived from a 60-item pool through expert review and cognitive interviews, and validated in a shape-classifier study ($N=200$) that supports both a unidimensional and a three-factor hierarchical structure, and (iii) a sentiment-classifier study ($N=200$) that varies explanations and stochasticity, and relates PSP to the correctness of users' predictions of system behavior, trust, subjective information processing awareness, and need for cognition. We find that PSP and prediction correctness capture distinct aspects of users' mental models and that both can diverge: PSP itself predicts correctness, explanations shift PSP but not correctness, and increased stochasticity degrades correctness without lowering PSP. PSP thus goes beyond existing objective and subjective measures and offers a principled foundation for designing transparent and trustworthy interactive systems.

Problem

• perceived system predictability
• human-computer interaction
• user perception
• trust
• uncertainty

Innovation

• perceived system predictability
• uncertainty theory
• scale validation
• human-AI interaction
• mental models