Score
Verification ensures a system is built correctly against specifications using unit tests, formal methods, static analysis, and model checking, while validation ensures the system meets user needs via integration tests, acceptance testing, simulation-based evaluation, and real-world A/B tests with defined success metrics.
This work addresses the fundamental tension in vulnerability detection for modern software systems: the limited scalability of formal methods versus the lack of formal guarantees in large language model (LLM)-based approaches. We systematically survey and, for the first time, comprehensively compare three paradigms—formal verification (model checking and theorem proving), LLM-driven analysis, and hybrid methods. We propose the first hybrid verification framework that jointly ensures formal correctness and leverages LLMs’ semantic understanding: LLMs guide invariant generation, defect localization, and verification goal pruning to intelligently augment formal workflows. We rigorously characterize the applicability boundaries and complementarity mechanisms among the three paradigms. Empirical evaluation on real-world programs demonstrates synergistic improvements across detection recall, verification strength, and analysis efficiency. Our framework establishes a new, principled pathway for high-assurance software analysis—balancing mathematical rigor with practical deployability.
This work addresses the challenge that counterexamples generated by formal verification often consist of numerous low-level Boolean variables, rendering them difficult for developers to interpret at the application-domain level. To bridge this gap, the paper proposes a novel hierarchical explanation method that integrates predicate relevance metrics with dependency graph analysis—a first-time fusion of these two techniques—to automatically extract human-readable, domain-oriented explanations from logical formulas. By leveraging formal modeling and a dedicated explanation-generation algorithm, the approach produces concise and semantically clear descriptions of failure causes across multiple case studies. Empirical results demonstrate that the method significantly outperforms existing techniques, offering effective support for fault localization in practical verification tasks.
This work addresses the challenges of model uncertainty and unpredictability in partially observable or black-box systems during runtime by proposing a unified theoretical framework that integrates epistemic logic with temporal logic. Leveraging automata theory, it systematically formalizes core concepts—including specification, diagnosis, opacity, and monitorability—and synthesizes lightweight online monitors through offline analysis. The approach is extended to real-time systems, resolving key issues related to their temporal semantics and algorithmic complexity. Furthermore, the study precisely characterizes the fundamental limits of runtime verification, thereby establishing a constructive and implementable foundation for practical deployment of monitoring mechanisms.
Existing code-level formal verification tools scale poorly to large-scale software, while mainstream unit-level verification relies heavily on manual effort, often missing critical defects. This paper proposes the “Unit Proof Framework” research agenda—the first systematic definition of a unit verification paradigm supporting automated decoupling and independent verification of code units. Methodologically, it integrates formal verification, program analysis, modular verification, and automated toolchain design, with deep alignment to industrial development practices (e.g., AWS workflows). Its core contributions include: (1) establishing a scalable, engineering-friendly unit verification methodology; (2) characterizing a taxonomy of key technical challenges; (3) overcoming bottlenecks inherent in manual verification; and (4) significantly improving early detection of code-level defects. Collectively, this work lays the theoretical foundation and provides a practical technical pathway for building high-assurance, deployable automated verification infrastructure.
Addressing the challenge of proving logical completeness relative to semantics in formal verification of software analysis systems (SAS), this paper introduces Representations—the first abstract metamodel tailored for SAS. Built on minimal assumptions, it uniformly characterizes syntax, semantics, and inference structures across diverse analysis systems, enabling systematic mapping between semantics and logic via metalinguistic modeling and structural induction. Its core innovation lies in decomposing completeness proofs into reusable, modular steps, thereby substantially reducing design and verification overhead for new SAS. Experiments demonstrate that Representations successfully reconstructs completeness proofs for multiple classical SAS and guides the development of two novel analysis systems—validating its effectiveness in simplifying proof construction, supporting system design, and ensuring theoretical rigor. (149 words)
Despite its efficacy in isolated projects, deductive verification has yet to achieve broad industrial adoption. To identify root barriers and key enablers, this paper conducts semi-structured interviews with 30 practitioners, followed by thematic analysis. We systematically uncover fundamental obstacles—including high proof maintenance overhead, limited automation, poor tool usability, and lack of workflow integration—as well as critical enabling factors. Diverging from prior work, we empirically establish *usability* and *workflow adaptability* as core dimensions governing adoption. Based on these findings, we propose three actionable improvement principles: (1) enhancing automation support for proof construction and evolution; (2) reducing proof maintenance burden through modularization and abstraction; and (3) deepening integration with IDEs and CI/CD pipelines. Our empirically grounded insights provide concrete, evidence-based guidance for tool developers, practitioners, and researchers—bridging the gap between academic verification techniques and engineering practice.
This work addresses the limited adoption of formal verification, which often requires expert-written annotations such as preconditions, postconditions, and loop invariants. To overcome this barrier, the authors propose a novel approach that leverages large language models (LLMs) in conjunction with assertions from test cases as static oracles to automatically generate Dafny verification annotations from code annotated with natural language comments. The method features an iterative refinement process guided by verifier feedback over multiple rounds and uniquely integrates multi-model LLM collaboration with a closed-loop verifier feedback mechanism. A VS Code plugin was developed to support practical deployment. Evaluated on 110 Dafny programs, the approach achieves a 98.2% annotation correctness rate within at most eight repair iterations. Empirical results highlight that proof-assistant-style annotation remains a key challenge for LLMs, while user feedback on the plugin was notably positive.
Existing statistical model checking methods suffer from insufficient theoretical foundations and limited verification reliability. This work establishes the first comprehensive probabilistic-logical formal framework for the SCAN statistical model checker, integrating probabilistic model checking, statistical hypothesis testing, and formal verification techniques to rigorously characterize the property verification process of complex systems. By unifying these complementary approaches within a sound theoretical basis, the proposed framework not only addresses the foundational gaps previously present in SCAN but also significantly enhances its rigor and applicability. Consequently, it provides a robust guarantee for the reliability of SCAN when applied to the verification of real-world systems.
This work proposes a novel paradigm that bridges the long-standing divide between testing and formal verification in traditional software validation, enabling them to synergistically enhance both efficiency and quality. Grounded in Design by Contract, the approach leverages the counterexample generation capability of SMT solvers to transform formal verification tools into an integrated engine for automated testing and repair. Within a unified framework, the method simultaneously achieves three key objectives: automatic generation of test cases for faulty programs, construction of regression test suites with full coverage for correct programs, and correctness-guaranteed program repair. This represents the first integration of verification, testing, and repair into a single cohesive methodology.
This work addresses the challenge of statically verifying semantic consistency between natural language business requirements and their code implementations. It proposes a two-stage, runtime-free approach: first leveraging large language models to extract structured rules from requirements while identifying ambiguous or contradictory statements, and then performing static code auditing based on this intermediate representation. By integrating natural language processing with static analysis, the method mitigates hallucination and context loss in large models through rule structuring, enabling requirement-aware early validation. Evaluated on an automotive cybersecurity case study, the approach successfully detects semantic deviations, offers a novel solution to the test oracle problem, and significantly enhances left-shifted verification capabilities.
This study addresses the limitations of existing SysML verification approaches, which are often tool-dependent and restricted to performance properties, lacking support for automated validation of behavioral and interface requirements. To overcome these shortcomings, this work proposes a tool-agnostic, automated verification workflow driven by SysML test cases, integrating UML Testing Profile and behavioral diagram constructs to enable unified validation of multidimensional attributes—including behavior, timing, and state responses. The methodology was developed through a mixed-methods research strategy combining literature review and stakeholder interviews, and its efficacy was empirically validated across two independent SysML toolchains. The approach not only transcends the constraints of conventional parametric methods but also enables automatic traceability of verification results back to the original model elements.