Understanding Bugs in Quantum Simulators: An Empirical Study

2026-03-24

AI Summary

This study addresses the critical lack of systematic investigation into faults in quantum simulators, which has left their reliability risks poorly understood. For the first time, we conduct an empirical analysis of 394 real-world defects across 12 widely used open-source quantum simulators. Through manual classification and root-cause tracing, we systematically characterize failure modes along multiple dimensions—including defect origins, manifestations, affected components, and detection mechanisms. Our findings reveal a prevalence of silent logical errors and critical failures stemming from classical infrastructure issues such as memory management and dependency compatibility, challenging the conventional testing paradigm that focuses narrowly on quantum-specific logic. Notably, most crashes and resource-related errors are reported post-deployment by users, whereas logical errors often produce incorrect outputs without triggering exceptions, offering crucial insights for advancing quantum software testing and verification methodologies.

Technology Category

• Machine Learning: Quantum Machine Learning
• Knowledge Representation and Reasoning: Qualitative Reasoning
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Web Mining and Content Analysis: Web data generation and simulation
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization

Abstract

Quantum simulators are a foundational component of the quantum software ecosystem. They are widely used to develop and debug quantum programs, validate compiler transformations, and support empirical claims about correctness and performance. In the absence of large-scale quantum hardware, simulator outputs are often treated as ground truth for algorithm development and system evaluation. However, quantum simulators also introduce unique implementation challenges. They must faithfully emulate quantum behavior while executing on classical hardware, requiring complex representations of quantum state evolution, operator composition, and noise modeling. Yet, we still lack a large-scale and in-depth study of failures in quantum simulators. To bridge this gap, this work presents a comprehensive empirical study of bugs in widely used open-source quantum simulators. We analyze 394 confirmed bugs from 12 simulators and manually categorize them based on root causes, failure manifestations, affected components, and discovery mechanisms. Our study reveals several key findings. First, bug discovery is largely user-driven, with most crashes, exceptions, and resource-related failures not detected by automated testing and identified after deployment. Second, logical correctness failures are widespread and often silent, producing plausible but incorrect outputs without triggering crashes or explicit error signals. Third, many critical failures originate in classical simulator infrastructure, such as memory management, indexing, configuration, and dependency compatibility, rather than in core quantum execution logic. These findings provide new insights into the reliability challenges of quantum simulators and highlight opportunities to improve testing and validation practices in the quantum software ecosystem.

Problem

• quantum simulators
• bugs
• empirical study
• software reliability
• failure analysis

Innovation

• quantum simulator bugs
• empirical study
• silent correctness failures
• classical infrastructure defects
• quantum software reliability