Investigating the originality of scientific papers across time and domain: A quantitative analysis

2025-12-03

AI Summary

Traditional bibliometric indicators fail to capture conceptual originality at the textual level of scientific papers. Method: We propose the Divergent Semantic Integration (DSI) metric, the first to combine SciBERT embeddings with network analysis to quantify cross-conceptual integration between abstracts and titles—thereby operationalizing textual originality. Contribution/Results: Empirical analysis of 51,200 papers shows DSI significantly predicts early citations—especially in single- and few-author papers—with pronounced disciplinary variation and temporal decay in predictive power. Adjusted R² in regression models reaches 0.103, demonstrating that textual originality is both measurable and predictively meaningful. This work establishes a novel paradigm for innovation assessment in science of science and provides a reusable computational framework for quantifying conceptual novelty from scholarly text.

Technology Category

• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Data Mining & Knowledge Management: Semantic Web
• Application Domains: Humanities & Computational Social Science

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Interdisciplinary science discovery with web data mining
• Social Networks and Social Media: Computational social science

Abstract

The study of creativity in science has long sought quantitative metrics capable of capturing the originality of the scientific insights contained within articles and other scientific works. In recent years, the field has witnessed a substantial expansion of research activity, enabled by advances in natural language processing and network analysis, and has utilised both macro- and micro-scale approaches with success. However, they often do not examine the text itself for evidence of originality. In this paper, we apply a computational measure correlating with originality from creativity science, Divergent Semantic Integration (DSI), to a set of 51,200 scientific abstracts and titles sourced from the Web of Science. To adapt DSI for application to scientific texts, we advance the original BERT method by incorporating SciBERT (a model trained on scientific corpora) into the computation of DSI. In our study, we observe that DSI plays a more pronounced role in the accrual of early citations for papers with fewer authors, varies substantially across subjects and research fields, and exhibits a declining correlation with citation counts over time. Furthermore, by modelling SciBERT- and BERT-DSI as predictors of the logarithm of 5-year citation counts alongside field, publication year, and the logarithm of author count, we find statistically significant relationships, with adjusted R-squared of 0.103 and 0.101 for BERT-DSI and SciBERT-DSI. Because existing scientometric measures rarely assess the originality expressed in textual content, DSI provides a valuable means of directly quantifying the conceptual originality embedded in scientific writing.

Problem

• Quantify originality in scientific papers using computational metrics
• Analyze how originality affects citation patterns across fields
• Develop a text-based method to measure conceptual novelty in research

Innovation

• Using Divergent Semantic Integration to measure originality
• Incorporating SciBERT to adapt DSI for scientific texts
• Quantifying conceptual originality directly from textual content