How to build an Open Science Monitor based on publications? A French perspective

2025-01-06

AI Summary

Assessing open science policies is hindered by difficulties in systematically leveraging publication data. Method: This study develops the first fully open, controllable, and reproducible metadata-driven monitoring framework, using journal articles, theses, and clinical trials as anchors to interlink research data, code, and software. It integrates open metadata sources, applies semantic standardization and mapping, aligns multi-source persistent identifiers (e.g., DOI, ISSN, Handle), and embeds FAIR governance throughout the workflow—enabling end-to-end traceability from publications to all scholarly outputs. Contribution/Results: The framework underpins France’s national open science monitoring system, enabling routine quantitative analysis of national open-access rates, green/gold pathway distributions, and cross-disciplinary openness trends. It significantly enhances the transparency, reproducibility, and systemic rigor of open science policy evaluation.

Technology Category

• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion
• Search and Optimization: Metareasoning and Metaheuristics
• Application Domains: Humanities & Computational Social Science

Application Category

• Security and Privacy: Data transparency and provenance
• Web Mining and Content Analysis: Web data provenance, reliability, and authenticity
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Many countries and institutions are striving to develop tools to monitor their open science policies. Since 2018, with the launch of its National Plan for Open Science, France has been progressively implementing a monitoring framework for its public policy, relying exclusively on reliable, open, and controlled data. Currently, this monitoring focuses on research outputs, particularly publications, as well as theses and clinical trials. Publications serve as a basis for analyzing other dimensions, including research data, code, and software. The metadata associated with publications is therefore particularly valuable, but the methodology for leveraging it raises several challenges. Here, we briefly outline how we have used this metadata to construct the French Open Science Monitor.

Problem

• Open Science Policy
• Publication Data
• Monitoring System

Innovation

• Open Science Policy
• Monitoring System
• Systematic Method