Validity in Design Science

2025-03-12

AI Summary

Design science has long lacked a unified definition and systematic validation mechanism for the validity of knowledge claims, undermining scholarly credibility and interdisciplinary communication. Method: This paper introduces the Design Science Validity Framework (DSVF)—the first comprehensive validity framework tailored to design science—defining validity rigorously and classifying knowledge claims into three primary types (principled, causal, and contextual) with corresponding subtypes. It establishes an operational framework architecture and an embedded process model that integrates validity assurance throughout the entire research lifecycle. Contribution/Results: Through framework engineering, claim taxonomy modeling, methodology meta-analysis, and cross-domain case validation, DSVF has successfully supported validity arguments in over ten design science studies and achieved self-validation of its own knowledge claims. The framework significantly enhances the rigor, reproducibility, and interdisciplinary communicability of design science outcomes.

Technology Category

• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Application Domains: Humanities & Computational Social Science
• Data Mining & Knowledge Management: Representing, Reasoning, and Using Provenance, Trust

Application Category

• Web Mining and Content Analysis: Web data provenance, reliability, and authenticity
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Security and Privacy: Data transparency and provenance

Abstract

Researchers must ensure that the claims about the knowledge produced by their work are valid. However, validity is neither well-understood nor consistently established in design science, which involves the development and evaluation of artifacts (models, methods, instantiations, and theories) to solve problems. As a result, it is challenging to demonstrate and communicate the validity of knowledge claims about artifacts. This paper defines validity in design science and derives the Design Science Validity Framework and a process model for applying it. The framework comprises three high-level claim and validity types-criterion, causal, and context-as well as validity subtypes. The framework guides researchers in integrating validity considerations into projects employing design science and contributes to the growing body of research on design science methodology. It also provides a systematic way to articulate and validate the knowledge claims of design science projects. We apply the framework to examples from existing research and then use it to demonstrate the validity of knowledge claims about the framework itself.

Problem

• Ensuring validity of knowledge claims in design science
• Developing a framework to validate design science artifacts
• Systematic validation of knowledge claims in design projects

Innovation

• Defines validity in design science
• Introduces Design Science Validity Framework
• Provides systematic validation of knowledge claims