Evidence for studying interactions between science and policy: An exploration of scholarly and policy references in Overton-indexed policy documents

2025-11-18

Journal of Data and Information Science

AI Summary

This study systematically examines the interaction between science and policy by analyzing citation patterns of academic outputs in policy documents. Leveraging the Overton global policy index database, it presents the first large-scale analysis of reference distributions across approximately 17.5 million policy documents, integrating multidimensional metadata—including publication year, source type, country, language, discipline, and policy topic—and employing descriptive statistics to characterize citation practices. The findings reveal that 7.7% of policy documents cite academic literature and 10.6% reference other policy documents, yet citation prevalence varies significantly across dimensions. These results underscore both the utility of Overton as a policy-metrics data source and its inherent coverage biases, thereby establishing a methodological foundation and practical guidance for future research on science–policy interactions.

Technology Category

• Application Domains: Humanities & Computational Social Science
• Data Mining & Knowledge Management: Other Foundations of Data Mining & Knowledge Management
• Natural Language Processing: Sentiment Analysis, Stylistic Analysis, and Argument Mining

Application Category

• Web Mining and Content Analysis: Interdisciplinary science discovery with web data mining
• Security and Privacy: Data transparency and provenance
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

ABSTRACT Purpose Overton, a global policy index, provides new opportunities to study the interactions between science and policy. This study aims to characterize the presence of scholarly and policy references in Overton-indexed policy documents and examine their distribution across key bibliographic dimensions, thereby assessing Overton’s potential as a data source for policy metrics. Design/methodology/approach We analyze a dataset of approximately 17.5 million policy documents from Overton, incorporating metadata such as publication year, policy source, country, language, subject area, and policy topic. Descriptive statistics are employed to assess the presence and distribution of reference data across these dimensions. Findings Overton indexes a substantial volume of policy documents and identifies considerable reference data within them: 7.7% of documents contain scholarly references and 10.6% contain policy references. However, the presence of references varies significantly across publications years, source types, countries, languages, subject areas, and policy topics, indicating coverage biases that may affect interpretations of policy impact. Research limitations The analysis is based on the Overton database as of June 2025. As Overton is regularly updated, the distribution patterns of indexed documents and references may evolve over time. Practical implications The findings offer insights into the opportunities and constraints of using Overton for investigating evidence-based policymaking and for assessing the policy uptake of research outputs in the context of research evaluation. Originality/Value This is the first large-scale study to systematically examine the distribution of reference data in Overton. It contributes a foundational understanding of this emerging source for policy metrics, highlighting both its potential applications and limitations, and underlining the importance of addressing current coverage imbalances.

Problem

• science-policy interactions
• policy citations
• scholarly references
• Overton database
• policy metrics

Innovation

• Overton
• policy metrics
• scholarly references
• evidence-based policymaking
• citation analysis