The Full-scale Assembly Simulation Testbed (FAST) Dataset

2024-03-13

arXiv.org

AI Summary

The VR assembly learning domain has long suffered from a lack of publicly available, large-scale, multimodal, and annotated datasets. Method: This work introduces and open-sources the first benchmark dataset specifically designed for VR assembly learning, comprising high-fidelity 6DoF motion tracking, eye-tracking, interaction logs, task timelines, assembly quality assessments, and demographic attributes from 108 participants assembling two full-scale structures in virtual environments. It establishes the first systematic linkage among user behavior, physiological responses, and cognitive state labels, underpinned by a standardized metadata schema. Contribution/Results: The dataset fills a critical gap in publicly accessible resources for VR skill acquisition research. It has already enabled empirical validation of downstream tasks—including user identification and cognitive load modeling—and provides a reproducible foundation for advancing VR-based educational assessment and adaptive training systems.

Technology Category

• Humans and AI: Game Design — Virtual Humans, NPCs and Autonomous Characters
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Computer Vision: Motion & Tracking

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Search and Retrieval-Augmented AI: Vertical and domain-specific search

Abstract

In recent years, numerous researchers have begun investigating how virtual reality (VR) tracking and interaction data can be used for a variety of machine learning purposes, including user identification, predicting cybersickness, and estimating learning gains. One constraint for this research area is the dearth of open datasets. In this paper, we present a new open dataset captured with our VR-based Full-scale Assembly Simulation Testbed (FAST). This dataset consists of data collected from 108 participants (50 females, 56 males, 2 non-binary) learning how to assemble two distinct full-scale structures in VR. In addition to explaining how the dataset was collected and describing the data included, we discuss how the dataset may be used by future researchers.

Problem

• Lack of open VR datasets for machine learning research
• Need for VR-based assembly simulation training data
• Providing a dataset for user behavior analysis in VR

Innovation

• VR-based Full-scale Assembly Simulation Testbed
• Open dataset from 108 participants
• Data for machine learning in VR