The QBF Gallery 2023

2026-04-17

AI Summary

This work addresses the lack of a unified and comprehensive benchmark suite for evaluating quantified Boolean formula (QBF) solvers. To this end, the authors construct a publicly available, integrative benchmark set by consolidating historically high-quality instances with newly contributed ones from the community. Leveraging a standardized evaluation framework, they conduct a systematic performance comparison of state-of-the-art QBF solvers. The resulting benchmark provides a more thorough and equitable platform for solver assessment, while the accompanying reproducible experimental results establish a solid foundation for future community initiatives such as the QBF Gallery. This contribution significantly advances the development and maturation of the QBF solver ecosystem.

Technology Category

• Constraint Satisfaction and Optimization: Solvers and Tools
• Knowledge Representation and Reasoning: Qualitative Reasoning
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs

Abstract

The QBF Gallery 2023, the last QBF evaluation event, continues the tradition to survey and document the state of the art in solving quantified Boolean formulas (QBFs). It provides a detailed overview by collecting newly developed solvers and formulas as benchmarks. This report documents the solvers and formulas submitted by the community and introduces a new, consolidated benchmark set that combines well-evaluated formulas with the submitted instances. The resulting formula set is made publicly available. With this benchmark set, we conduct a comparative analysis of the submitted solvers and publicly available solvers, assessing their performance and current capabilities. In addition, we report on the present status of the QBF Gallery and discuss ideas and directions for future editions to further support research and benchmarking within the QBF community.

Problem

• QBF
• quantified Boolean formulas
• benchmarking
• solver evaluation
• QBF Gallery

Innovation

• QBF
• benchmarking
• solver evaluation
• quantified Boolean formulas
• comparative analysis