CLAd-VR: Cognitive Load-based Adaptive Training for Machining Tasks in Virtual Reality

2025-10-06

AI Summary

To address the mismatch between static instruction and learners’ cognitive load in manufacturing VR training, this study proposes the first real-time EEG-based adaptive VR training system. Using a wearable EEG device, the system captures neural signals and employs a lightweight LSTM model to dynamically classify cognitive load at millisecond resolution. This classification drives multimodal feedback—such as animations, voice prompts, and virtual hand demonstrations—within the VR environment to adjust task difficulty and instructional pacing in real time. Its key innovation lies in the first integration of online EEG-based cognitive load decoding with a closed-loop VR training framework, establishing an end-to-end personalized adaptive system. Experimental results demonstrate that the system significantly reduces cognitive overload while improving operational accuracy and knowledge retention. This work establishes a scalable, human-centered training paradigm for smart manufacturing.

Technology Category

• Cognitive Modeling & Cognitive Systems: Adaptive Behavior
• Machine Learning: Hardware-aware ML
• Intelligent Robots: Cognitive Robotics

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

With the growing need to effectively support workforce upskilling in the manufacturing sector, virtual reality is gaining popularity as a scalable training solution. However, most current systems are designed as static, step-by-step tutorials and do not adapt to a learner's needs or cognitive load, which is a critical factor in learning and longterm retention. We address this limitation with CLAd-VR, an adaptive VR training system that integrates realtime EEG-based sensing to measure the learner's cognitive load and adapt instruction accordingly, specifically for domain-specific tasks in manufacturing. The system features a VR training module for a precision drilling task, designed with multimodal instructional elements including animations, text, and video. Our cognitive load sensing pipeline uses a wearable EEG device to capture the trainee's neural activity, which is processed through an LSTM model to classify their cognitive load as low, optimal, or high in real time. Based on these classifications, the system dynamically adjusts task difficulty and delivers adaptive guidance using voice guidance, visual cues, or ghost hand animations. This paper introduces CLAd-VR system's architecture, including the EEG sensing hardware, real-time inference model, and adaptive VR interface.

Problem

• Adapts VR training to learner's cognitive load using EEG sensing
• Addresses static tutorials by dynamically adjusting task difficulty
• Provides real-time adaptive guidance for manufacturing skill training

Innovation

• Uses EEG-based real-time cognitive load sensing
• Adapts task difficulty and guidance dynamically
• Integrates LSTM model for cognitive load classification