validation and verification

Verification ensures a system is built correctly against specifications using unit tests, formal methods, static analysis, and model checking, while validation ensures the system meets user needs via integration tests, acceptance testing, simulation-based evaluation, and real-world A/B tests with defined success metrics.

validationandverification

12-Month Skill Trend

Momentum and market value over time
Trending
Score
+20 in 12 mo
96
12 mo agoNow
Career
Value
+$12K in 12 mo
$42K/year
12 mo agoNow

Recommended Survey Paper

Quick overview of the field
View more

Must-Read Papers

Most classic and influential ideas
View more

This work addresses the challenge that counterexamples generated by formal verification often consist of numerous low-level Boolean variables, rendering them difficult for developers to interpret at the application-domain level. To bridge this gap, the paper proposes a novel hierarchical explanation method that integrates predicate relevance metrics with dependency graph analysis—a first-time fusion of these two techniques—to automatically extract human-readable, domain-oriented explanations from logical formulas. By leveraging formal modeling and a dedicated explanation-generation algorithm, the approach produces concise and semantically clear descriptions of failure causes across multiple case studies. Empirical results demonstrate that the method significantly outperforms existing techniques, offering effective support for fault localization in practical verification tasks.

application domain modelcounterexample interpretationformal verification

This work addresses the challenges of model uncertainty and unpredictability in partially observable or black-box systems during runtime by proposing a unified theoretical framework that integrates epistemic logic with temporal logic. Leveraging automata theory, it systematically formalizes core concepts—including specification, diagnosis, opacity, and monitorability—and synthesizes lightweight online monitors through offline analysis. The approach is extended to real-time systems, resolving key issues related to their temporal semantics and algorithmic complexity. Furthermore, the study precisely characterizes the fundamental limits of runtime verification, thereby establishing a constructive and implementable foundation for practical deployment of monitoring mechanisms.

black-box systemsmonitoringpartial observability

A Unit Proofing Framework for Code-level Verification: A Research Agenda

Oct 18, 2024
PC
Paschal C. Amusuo
🏛️ Purdue University | Michigan State University

Existing code-level formal verification tools scale poorly to large-scale software, while mainstream unit-level verification relies heavily on manual effort, often missing critical defects. This paper proposes the “Unit Proof Framework” research agenda—the first systematic definition of a unit verification paradigm supporting automated decoupling and independent verification of code units. Methodologically, it integrates formal verification, program analysis, modular verification, and automated toolchain design, with deep alignment to industrial development practices (e.g., AWS workflows). Its core contributions include: (1) establishing a scalable, engineering-friendly unit verification methodology; (2) characterizing a taxonomy of key technical challenges; (3) overcoming bottlenecks inherent in manual verification; and (4) significantly improving early detection of code-level defects. Collectively, this work lays the theoretical foundation and provides a practical technical pathway for building high-assurance, deployable automated verification infrastructure.

Automating unit proofing to reduce manual errorsEarly detection of implementation defects in verificationEnsuring code-level correctness in large-scale software

Representations

Oct 13, 2025
PB
Paul Brunet
🏛️ Université Paris-Est Créteil Val de Marne

Addressing the challenge of proving logical completeness relative to semantics in formal verification of software analysis systems (SAS), this paper introduces Representations—the first abstract metamodel tailored for SAS. Built on minimal assumptions, it uniformly characterizes syntax, semantics, and inference structures across diverse analysis systems, enabling systematic mapping between semantics and logic via metalinguistic modeling and structural induction. Its core innovation lies in decomposing completeness proofs into reusable, modular steps, thereby substantially reducing design and verification overhead for new SAS. Experiments demonstrate that Representations successfully reconstructs completeness proofs for multiple classical SAS and guides the development of two novel analysis systems—validating its effectiveness in simplifying proof construction, supporting system design, and ensuring theoretical rigor. (149 words)

Creating formal meta-model for software analysis systemsDeveloping verification frameworks for automated systems analysisEstablishing logic completeness with respect to semantics

Despite its efficacy in isolated projects, deductive verification has yet to achieve broad industrial adoption. To identify root barriers and key enablers, this paper conducts semi-structured interviews with 30 practitioners, followed by thematic analysis. We systematically uncover fundamental obstacles—including high proof maintenance overhead, limited automation, poor tool usability, and lack of workflow integration—as well as critical enabling factors. Diverging from prior work, we empirically establish *usability* and *workflow adaptability* as core dimensions governing adoption. Based on these findings, we propose three actionable improvement principles: (1) enhancing automation support for proof construction and evolution; (2) reducing proof maintenance burden through modularization and abstraction; and (3) deepening integration with IDEs and CI/CD pipelines. Our empirically grounded insights provide concrete, evidence-based guidance for tool developers, practitioners, and researchers—bridging the gap between academic verification techniques and engineering practice.

Developing recommendations for practitioners and tool builders to improve adoptionIdentifying underexplored obstacles like proof maintenance and usability issuesInvestigating barriers to mainstream adoption of deductive verification methods

Latest Papers

What's happening recently
View more

This work addresses the limited adoption of formal verification, which often requires expert-written annotations such as preconditions, postconditions, and loop invariants. To overcome this barrier, the authors propose a novel approach that leverages large language models (LLMs) in conjunction with assertions from test cases as static oracles to automatically generate Dafny verification annotations from code annotated with natural language comments. The method features an iterative refinement process guided by verifier feedback over multiple rounds and uniquely integrates multi-model LLM collaboration with a closed-loop verifier feedback mechanism. A VS Code plugin was developed to support practical deployment. Evaluated on 110 Dafny programs, the approach achieves a 98.2% annotation correctness rate within at most eight repair iterations. Empirical results highlight that proof-assistant-style annotation remains a key challenge for LLMs, while user feedback on the plugin was notably positive.

Dafnyformal specificationLLMs

Existing statistical model checking methods suffer from insufficient theoretical foundations and limited verification reliability. This work establishes the first comprehensive probabilistic-logical formal framework for the SCAN statistical model checker, integrating probabilistic model checking, statistical hypothesis testing, and formal verification techniques to rigorously characterize the property verification process of complex systems. By unifying these complementary approaches within a sound theoretical basis, the proposed framework not only addresses the foundational gaps previously present in SCAN but also significantly enhances its rigor and applicability. Consequently, it provides a robust guarantee for the reliability of SCAN when applied to the verification of real-world systems.

formal foundationsSCANstatistical model checking

This work proposes a novel paradigm that bridges the long-standing divide between testing and formal verification in traditional software validation, enabling them to synergistically enhance both efficiency and quality. Grounded in Design by Contract, the approach leverages the counterexample generation capability of SMT solvers to transform formal verification tools into an integrated engine for automated testing and repair. Within a unified framework, the method simultaneously achieves three key objectives: automatic generation of test cases for faulty programs, construction of regression test suites with full coverage for correct programs, and correctness-guaranteed program repair. This represents the first integration of verification, testing, and repair into a single cohesive methodology.

automatic program repairformal proofsoftware maintenance

This work addresses the challenge of statically verifying semantic consistency between natural language business requirements and their code implementations. It proposes a two-stage, runtime-free approach: first leveraging large language models to extract structured rules from requirements while identifying ambiguous or contradictory statements, and then performing static code auditing based on this intermediate representation. By integrating natural language processing with static analysis, the method mitigates hallucination and context loss in large models through rule structuring, enabling requirement-aware early validation. Evaluated on an automotive cybersecurity case study, the approach successfully detects semantic deviations, offers a novel solution to the test oracle problem, and significantly enhances left-shifted verification capabilities.

business logic validationcode compliancenatural-language requirements

This study addresses the limitations of existing SysML verification approaches, which are often tool-dependent and restricted to performance properties, lacking support for automated validation of behavioral and interface requirements. To overcome these shortcomings, this work proposes a tool-agnostic, automated verification workflow driven by SysML test cases, integrating UML Testing Profile and behavioral diagram constructs to enable unified validation of multidimensional attributes—including behavior, timing, and state responses. The methodology was developed through a mixed-methods research strategy combining literature review and stakeholder interviews, and its efficacy was empirically validated across two independent SysML toolchains. The approach not only transcends the constraints of conventional parametric methods but also enables automatic traceability of verification results back to the original model elements.

behavioral propertiesinterface propertiesmodel verification

Hot Scholars

MZ

Martin Zimmermann

Aalborg University
computer scienceverificationautomata theorytemporal logics
GK

Guy Katz

The Hebrew University of Jerusalem
VerificationSoftware Engineering
LC

Lucas C. Cordeiro

Professor of Computer Science, University of Manchester
Formal MethodsAutomated VerificationSoftware TestingProgram Synthesis
BF

Bernd Finkbeiner

Professor of Computer Science, CISPA Helmholtz Center for Information Security
Reactive SystemsVerificationSynthesisTemporal Logic
JP

Joost-Pieter Katoen

Distinguished Professor of Computer Science, RWTH Aachen University and University of Twente
formal methodsmodel checkingconcurrency theoryprobabilistic programming