Emotion Detection on User Front-Facing App Interfaces for Enhanced Schedule Optimization: A Machine Learning Approach

2025-06-23

AI Summary

This study addresses the insufficient modeling of the relationship between users’ affective states and productivity in calendar applications. We propose a dual-channel affect-aware framework that jointly leverages ECG-derived physiological signals—specifically heart rate variability (HRV) features—and computer interaction behaviors—including mouse movement, clicks, and keystrokes—for multimodal affect recognition. Methodologically, a GRU network models temporal ECG dynamics, achieving 84.38% accuracy in valence prediction; meanwhile, machine learning classifiers trained on mouse behavioral features attain 90% accuracy in emotion classification, significantly outperforming unimodal baselines. The framework enables real-time adaptation of calendar interface logic and time-reminder strategies based on inferred affective and stress states. Empirical evaluation demonstrates improved user focus and task engagement. This work provides a deployable technical pathway and empirical validation for affectively intelligent human–computer interaction in productivity tools.

Technology Category

• Cognitive Modeling & Cognitive Systems: Affective Computing
• Planning, Routing, and Scheduling: Activity and Plan Recognition
• Humans and AI: Human-Computer Interaction

Application Category

• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

Human-Computer Interaction (HCI) has evolved significantly to incorporate emotion recognition capabilities, creating unprecedented opportunities for adaptive and personalized user experiences. This paper explores the integration of emotion detection into calendar applications, enabling user interfaces to dynamically respond to users' emotional states and stress levels, thereby enhancing both productivity and engagement. We present and evaluate two complementary approaches to emotion detection: a biometric-based method utilizing heart rate (HR) data extracted from electrocardiogram (ECG) signals processed through Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) neural networks to predict the emotional dimensions of Valence, Arousal, and Dominance; and a behavioral method analyzing computer activity through multiple machine learning models to classify emotions based on fine-grained user interactions such as mouse movements, clicks, and keystroke patterns. Our comparative analysis, from real-world datasets, reveals that while both approaches demonstrate effectiveness, the computer activity-based method delivers superior consistency and accuracy, particularly for mouse-related interactions, which achieved approximately 90% accuracy. Furthermore, GRU networks outperformed LSTM models in the biometric approach, with Valence prediction reaching 84.38% accuracy.

Problem

• Detect user emotions from app interfaces for better scheduling
• Compare biometric and behavioral emotion detection methods
• Enhance productivity via emotion-responsive calendar applications

Innovation

• Biometric emotion detection using ECG and GRU networks
• Behavioral emotion analysis via mouse and keyboard interactions
• Comparative evaluation showing GRU outperforms LSTM models