TPIFM: A Task-Aware Model for Evaluating Perceptual Interaction Fluency in Remote AR Collaboration

2026-03-10

AI Summary

This study addresses the degradation of perceived interaction fluency (PIF) in remote augmented reality collaboration caused by network latency and stuttering, noting that existing approaches overlook how task characteristics influence tolerance to such impairments. For the first time, the authors integrate task-specific just-noticeable differences (JNDs) with the free energy principle to analyze PIF performance across diverse tasks under various network impairments. Through subjective experiments and measurements of average response times, they develop a Task-aware Perceived Interaction Fluency Model (TPIFM). This model accurately predicts PIF levels for different tasks under varying network conditions, uncovering the intrinsic mechanisms by which task type modulates tolerance to interaction disruptions. The findings provide both theoretical grounding and practical guidance for designing adaptive remote AR systems and optimizing user experience.

Technology Category

• Cognitive Modeling & Cognitive Systems: Social Cognition And Interaction
• Humans and AI: Interaction Techniques and Devices
• Knowledge Representation and Reasoning: Preferences

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Remote Collaborative Augmented Reality (RCAR) enables geographically distributed users to collaborate by integrating virtual and physical environments. However, because RCAR relies on real-time transmission, it is susceptible to delay and stalling impairments under constrained network conditions. Perceptual interaction fluency (PIF), defined as the perceived pace and responsiveness of collaboration, is influenced not only by physical network impairments but also by intrinsic task characteristics. These characteristics can be interpreted as the task-specific just-noticeable difference (JND), i.e., the maximal tolerable temporal responsiveness before PIF degrades. When the average response time (ART), measured as the mean time per operation from receiving collaborator feedback to initiating the next action, falls within the JND, PIF is generally sustained, whereas values exceeding it indicate disruption. Tasks differ in their JNDs, reflecting distinct temporal responsiveness demands and sensitivities to impairments. From the perspective of the Free Energy Principle (FEP), tasks with lower JNDs impose stricter temporal prediction demands, making PIF more vulnerable to impairments, whereas higher JNDs allow greater tolerance. On this basis, we classify RCAR tasks by JND and evaluate their PIF through controlled subjective experiments under delay, stalling, and hybrid conditions. Building on these findings, we propose the Task-Aware Perceptual Interaction Fluency Model (TPIFM). Experimental results show that TPIFM accurately assesses PIF under network impairments, providing guidance for adaptive RCAR design and user experience optimization under network constraints.

Problem

• Remote Collaborative Augmented Reality
• Perceptual Interaction Fluency
• Task Characteristics
• Network Impairments
• Just-Noticeable Difference

Innovation

• Perceptual Interaction Fluency
• Task-Aware Modeling
• Just-Noticeable Difference (JND)
• Remote Collaborative AR
• Free Energy Principle