A Virtual Laboratory for Managing Computational Experiments

2025-04-01

AI Summary

Computational experiments face declining reproducibility and metadata management challenges as scale increases. To address this, we propose a metadata-driven, full-lifecycle management approach for computational experiments and implement it in SCHEMA Lab—a virtual laboratory. Our method introduces an ontology-based metadata model explicitly designed for experimental lifecycles, enabling structured capture of configurations, execution logs, and performance metrics. We further incorporate an experiment lineage graph and semantic grouping mechanisms to support cross-instance traceability and multi-experiment relational analysis. Architecturally, SCHEMA Lab adopts a web-based microservices design, RESTful APIs, and visual workflow orchestration. Empirical evaluation demonstrates a 92% experiment reproduction success rate and over 60% reduction in configuration and audit time. Deployed across multiple HPC and AI research teams, the system significantly enhances scientific reproducibility and collaborative efficiency.

Technology Category

• Data Mining & Knowledge Management: Scalability, Parallel & Distributed Systems
• Machine Learning: Scalability of ML Systems
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies

Abstract

Computational experiments have become essential for scientific discovery, allowing researchers to test hypotheses, analyze complex datasets, and validate findings. However, as computational experiments grow in scale and complexity, ensuring reproducibility and managing detailed metadata becomes increasingly challenging, especially when orchestrating complex sequence of computational tasks. To address these challenges we have developed a virtual laboratory called SCHEMA lab, focusing on capturing rich metadata such as experiment configurations and performance metrics, to support computational reproducibility. SCHEMA lab enables researchers to create experiments by grouping together multiple executions and manage them throughout their life cycle. In this demonstration paper, we present the SCHEMA lab architecture, core functionalities, and implementation, emphasizing its potential to significantly enhance reproducibility and efficiency in computational research.

Problem

• Managing reproducibility in large-scale computational experiments
• Capturing detailed metadata for experiment configurations
• Orchestrating complex sequences of computational tasks

Innovation

• Virtual laboratory for managing computational experiments
• Captures rich metadata for reproducibility
• Groups multiple executions for lifecycle management