Towards SMT Solver Stability via Input Normalization

2024-10-29

AI Summary

SMT solvers exhibit high sensitivity to minor input perturbations, causing drastic fluctuations in solving performance and severely undermining usability. To address this, we formally define “perfect normalization” for SMT inputs—the first such rigorous characterization—and propose an efficient approximation algorithm that achieves high coverage and deterministic canonical mapping. Our method integrates input normalization, equivalence-aware formula identification, and mutation-driven validation. Evaluated on SMT-LIB benchmarks and large-scale mutation suites, it reduces solver runtime variance by over 40% on average while incurring negligible overhead (<1% of total solving time). Key contributions include: (1) the first computationally tractable theoretical framework for SMT input normalization; (2) the first lightweight, end-to-end integrable normalization implementation deployable within existing solver pipelines; and (3) empirical validation demonstrating substantial improvements in solving stability and formula reusability across diverse benchmarks.

Technology Category

• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories
• Search and Optimization: Heuristic Search
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Normalization, clustering, classification, and summarization of Web text
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization

Abstract

In many applications, SMT solvers are utilized to solve similar or identical tasks over time. Significant variations in performance due to small changes in the input are not uncommon and lead to frustration for users. This sort of stability problem represents an important usability challenge for SMT solvers. We introduce an approach for mitigating the stability problem based on normalizing solver inputs. We show that a perfect normalizing algorithm exists but is computationally expensive. We then describe an approximate algorithm and evaluate it on a set of benchmarks from related work, as well as a large set of benchmarks sampled from SMT-LIB. Our evaluation shows that our approximate normalizer reduces runtime variability with minimal overhead and is able to normalize a large class of mutated benchmarks to a unique normal form.

Problem

• Addressing SMT solver performance instability from input variations
• Mitigating usability issues via input normalization techniques
• Reducing runtime variability with efficient approximate normalization

Innovation

• Input normalization for SMT solver stability
• Approximate algorithm reduces runtime variability
• Normalizes mutated benchmarks to unique form