Measuring a moving target -- Innovation studies in practice

2025-03-05

AI Summary

Emerging scientific infrastructures—such as data platforms, interdisciplinary centers, and open science practices—challenge traditional bibliometric and innovation metrics. Method: Employing a dual-track approach combining empirical measurement experiments with philosophical reflection, the study integrates scientometrics, systems modeling, cross-case comparison, and epistemological critique to re-examine the early-2000s innovation studies research program. Contribution/Results: The work reconstructs the ontological and epistemological foundations of knowledge-base metrics, establishing a legitimacy framework for innovation measurement under dynamic objectives. Findings confirm the enduring explanatory power of the original program’s core tenets; reveal how the three emerging practices critically undermine existing indicator systems; and argue that quantitative innovation research must be embedded within a reflexive meta-level. Its principal contribution is the first integrated paradigm for innovation metrics that coherently unites empirical measurement with philosophical reflection.

Technology Category

• Game Theory and Economic Paradigms: Other Foundations of Game Theory & Economic Paradigms
• Data Mining & Knowledge Management: Other Foundations of Data Mining & Knowledge Management
• Search and Optimization: Metareasoning and Metaheuristics

Application Category

• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Web Mining and Content Analysis: Web measurements
• Security and Privacy: Large-scale security measurements

Abstract

This paper pays a tribute to Loet's work in a specific way. More than 20 years ago Loet Leydesdorff and myself designed a programme for future innovation studies 'Measuring the knowledge base - a programme of innovation studies'. Although, the funding programme we envisioned eventually did not materialise, the proposal text set out the main lines of our research collaboration over the coming decades. This paper revisits main statements of this programme and discusses their remaining validity in the light of more recent research. Core to the Leydesdorff and Scharnhorst text was a system-theoretical, evolutionary perspective on science dynamics, newly emerging structures and phenomena and addressed the question to which extent they could be meaningful studied using quantitative approaches. This paper looks into three cases - all examples of newly emerging institutional structures and related practices in science. They are all located at the interface between research and research infrastructures. While discussing how the programmatic ideas written up at the beginning of the 2000s still informs measurement attempts in those three cases, the paper also touches upon questions of epistemological foundations of quantitative studies in general. The main conclusion is that combining measurement experiments with philosophical reflection remains important.

Problem

• Evaluates validity of past innovation studies program.
• Explores quantitative approaches for studying science dynamics.
• Assesses epistemological foundations of quantitative studies.

Innovation

• System-theoretical, evolutionary perspective on science dynamics
• Quantitative approaches to study emerging structures
• Combining measurement experiments with philosophical reflection