An Active Inference Model of Mouse Point-and-Click Behaviour

2025-10-16

AI Summary

This work addresses the challenges of latency compensation and target difficulty adaptation in mouse-pointing tasks within human–computer interaction (HCI). We propose the first continuous-space user behavior model grounded in full-probabilistic active inference (AIF), wherein the user is formalized as an agent minimizing expected free energy. The model integrates perceptual preference distributions, dynamic latency modeling, and probabilistic predictive coding, enabling natural, parameter-free latency-aware compensation and endpoint variability regulation. Our key contributions are threefold: (1) the first application of the complete AIF framework to continuous HCI pointing tasks, supporting end-to-end trajectory generation; (2) automatic adaptation to varying Fitts’ law task difficulties without manual tuning; and (3) faithful reproduction of human-like click accuracy and endpoint dispersion patterns. Empirical evaluation confirms both behavioral plausibility and computational efficiency.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• User Modeling, Personalization and Recommendation: Psychology-informed user models and recommender systems
• Economics, Online Markets and Human Computation: Research challenges in human and human-AI computation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

We explore the use of Active Inference (AIF) as a computational user model for spatial pointing, a key problem in Human-Computer Interaction (HCI). We present an AIF agent with continuous state, action, and observation spaces, performing one-dimensional mouse pointing and clicking. We use a simple underlying dynamic system to model the mouse cursor dynamics with realistic perceptual delay. In contrast to previous optimal feedback control-based models, the agent's actions are selected by minimizing Expected Free Energy, solely based on preference distributions over percepts, such as observing clicking a button correctly. Our results show that the agent creates plausible pointing movements and clicks when the cursor is over the target, with similar end-point variance to human users. In contrast to other models of pointing, we incorporate fully probabilistic, predictive delay compensation into the agent. The agent shows distinct behaviour for differing target difficulties without the need to retune system parameters, as done in other approaches. We discuss the simulation results and emphasize the challenges in identifying the correct configuration of an AIF agent interacting with continuous systems.

Problem

• Modeling mouse pointing and clicking using Active Inference
• Incorporating perceptual delay and probabilistic delay compensation
• Generating human-like cursor movements without parameter retuning

Innovation

• Active Inference agent minimizes Expected Free Energy
• Probabilistic predictive delay compensation for cursor control
• No retuning needed for varying target difficulties