LUIDA: Large-scale Unified Infrastructure for Digital Assessments based on Commercial Metaverse Platform

2025-04-24

AI Summary

Current metaverse online experimentation suffers from fragmented workflows and disjointed tools, resulting in high development barriers and poor reproducibility. To address this, we present the first large-scale, unified digital evaluation infrastructure tailored for the commercial metaverse platform Cluster. Our approach introduces a metaverse-native, integrated experimental framework that unifies automated virtual environment orchestration, cross-domain reusable experiment template engines, and a lightweight VR development interface. Empirical evaluation demonstrates high system usability (SUS = 73.75) and low cognitive load (NASA-TLX = 24.11). Three large-scale experiments—each involving over 100 participants—successfully replicated seminal findings across disciplines, confirming the framework’s interdisciplinary validity and strong reproducibility. By significantly reducing VR experiment development complexity, our infrastructure provides a standardized, scalable methodological foundation for empirical metaverse research.

Technology Category

• Humans and AI: Game Design — Virtual Humans, NPCs and Autonomous Characters
• Application Domains: Humanities & Computational Social Science
• Search and Optimization: Metareasoning and Metaheuristics

Application Category

• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Systems and Infrastructure for Web, Mobile and WoT: Virtualization and resource management in Web systems and infrastructures

Abstract

Online experiments using metaverse platforms have gained significant traction in Human-Computer Interaction and Virtual Reality (VR) research. However, current research workflows are highly fragmented, as researchers must use separate tools for system implementation, participant recruitment, experiment execution, and data collection, reducing consistency and increasing workload. We present LUIDA (Large-scale Unified Infrastructure for Digital Assessments), a metaverse-based framework that integrates these fragmented processes. LUIDA automatically allocates interconnected virtual environments for parallel experiment execution and provides implementation templates adaptable to various VR research domains, requiring minimal metaverse development expertise. Our evaluation included two studies using a prototype built on Cluster, the commercial metaverse platform. First, VR researchers using LUIDA to develop and run experiments reported high usability scores (SUS: 73.75) and moderate workload (NASA-TLX: 24.11) for overall usage, with interviews confirming streamlined workflows compared to traditional laboratory experiments. Second, we conducted three replicated experiments with public Cluster users, each recruiting approximately 200 participants within one week. These experiments produced results that closely matched the original studies, validating the experimental integrity of LUIDA across research domains. After technical refinements, we plan to release LUIDA as an open platform, providing a standardized protocol to improve research efficiency and experimental reproducibility in VR studies.

Problem

• Integrates fragmented tools for VR research workflows
• Automates virtual environment allocation for parallel experiments
• Enhances research efficiency and experimental reproducibility

Innovation

• Unified metaverse framework for digital assessments
• Automated virtual environment allocation for experiments
• Open platform for VR research efficiency