Robust Evidence for Declining Disruptiveness: Assessing the Role of Zero-Backward-Citation Works

2025-02-28

AI Summary

This study addresses Holst et al.’s critique of Park et al.’s claim that scientific disruptiveness is declining, specifically questioning whether the observed decline stems from including zero backward-citation (0-bcite) publications. Method: We rigorously replicate and extend Park et al.’s analysis using bibliometric statistics, multivariate regression, Monte Carlo simulations, and cross-dataset validation across scholarly articles and patents with multiple disruption metrics. Contribution/Results: First, applying Park et al.’s own model, we demonstrate that the decline in disruptiveness remains highly significant (p < 0.001) even after excluding all 0-bcite publications—its magnitude comparable to major paradigm shifts. Second, we identify methodological biases and metadata quality issues in the critical study. Collectively, our findings robustly confirm that the decline in scientific disruptiveness is a genuine, empirically grounded phenomenon—not an artifact of data or measurement choices.

Technology Category

• Reasoning under Uncertainty: Causality
• Cognitive Modeling & Cognitive Systems: Analogy
• Application Domains: Humanities & Computational Social Science

Application Category

• Web Mining and Content Analysis: Interdisciplinary science discovery with web data mining
• Social Networks and Social Media: Computational social science
• Security and Privacy: Data transparency and provenance

Abstract

We respond to Holst et al.'s (HATWG) critique that the observed decline in scientific disruptiveness demonstrated in Park et al. (PLF) stems from including works with zero backward citations (0-bcites). Applying their own advocated dataset, metric, and exclusion criteria, we demonstrate statistically and practically significant declines in disruptiveness that equal major benchmark transformations in science. Notably, we show that HATWG's own regression model -- designed specifically to address their concerns about 0-bcite works -- reveals highly significant declines for both papers (p<0.001) and patents (p<0.001), a finding they neither acknowledge nor interpret. Their critique is undermined by methodological deficiencies, including reliance on visual inspection without statistical assessment, and severe data quality issues in their SciSciNet dataset, which contains nearly three times more 0-bcite papers than our original data. HATWG's departure from established scientometric practices -- notably their inclusion of document types and fields known for poor metadata quality -- invalidates their conclusions. Monte Carlo simulations and additional analyses using multiple disruptiveness measures across datasets further validate the robustness of the declining trend. Our findings collectively demonstrate that the observed decline in disruptiveness is not an artifact of 0-bcite works but represents a substantive change in scientific and technological innovation patterns.

Problem

• Decline in scientific disruptiveness due to zero-backward-citation works.
• Methodological flaws in critique by Holst et al. undermine their conclusions.
• Robust evidence supports declining disruptiveness in science and technology.

Innovation

• Statistical analysis of disruptiveness decline
• Monte Carlo simulations validate trends
• Multiple disruptiveness measures across datasets