The Data Dilemma: Authors' Intentions and Recognition of Research Data in Educational Technology Research

2025-06-05

AI Summary

In EdTech research, software and qualitative data are frequently overlooked by authors, leading to systematic biases in data identification and classification—revealing a critical non-technical barrier to FAIR implementation: deficient data awareness. This study employs a mixed-methods approach: (1) quantitative analysis of alignment between author-declared data types and actual data reported in submissions to the DELFI 2024 conference (via EasyChair); (2) qualitative coding to uncover implicit data categories; and (3) bibliometric analysis to map current data practices. Results indicate that over 40% of empirically used data remain unacknowledged or undeclared by authors, with software and qualitative data exhibiting the lowest declaration rates. This is the first systematic investigation to expose domain-specific data cognition gaps in EdTech. The findings inform the development of EdTech-tailored data publication guidelines and targeted research data literacy training frameworks, thereby advancing FAIR compliance in educational technology scholarship.

Technology Category

• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy
• Application Domains: Humanities & Computational Social Science
• Philosophy and Ethics of AI: Bias, Fairness & Equity

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Web Mining and Content Analysis: Web data quality in the era of algorithmically-generated content
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

Educational Technology (EdTec) research is conducted by multiple disciplines, some of which annually meet at the DELFI conference. Due to the heterogeneity of involved researchers and communities, it is our goal to identify categories of research data overseen in the context of EdTec research. Therefore, we analyze the author's perspective provided via EasyChair where authors specified whether they had research data to share. We compared this information with an analysis of the submitted articles and the contained research data. We found that not all research data was recognized as such by the authors, especially software and qualitative data, indicating a prevailing lack of awareness, and other potential barriers. In addition, we analyze the 2024 DELFI proceedings to learn what kind of data was subject to research, and where it is published. This work has implications for training future generations of EdTec researchers. It further stresses the need for guidelines and recognition of research data publications (particularly software, and qualitative data).

Problem

• Identifying overlooked research data categories in EdTec due to disciplinary heterogeneity
• Assessing author awareness gaps in recognizing shared research data (e.g., software)
• Analyzing DELFI proceedings to improve data publication guidelines and training

Innovation

• Analyze author perspectives via EasyChair submissions
• Compare submitted articles with research data
• Examine DELFI proceedings for data types