The Case of the Mysterious Citations

2026-02-05

AI Summary

This study addresses the growing prevalence of unverifiable “mystery citations” in academic publishing, exacerbated by widespread non-disclosure of AI-generated references contrary to conference policies, revealing critical flaws in citation validation mechanisms. We develop an automated analysis pipeline integrating textual comparison, metadata verification, and policy compliance auditing to systematically evaluate citation accuracy across papers from four premier high-performance computing conferences between 2021 and 2025. Our findings quantitatively demonstrate that, by 2025, all examined venues exhibited mystery citations affecting 2%–6% of publications, accompanied by a marked increase in erroneous titles and author information. The results confirm the ineffectiveness of current policy enforcement and underscore the urgent need for updated citation standards and enhanced technical oversight.

Technology Category

• Data Mining & Knowledge Management: Representing, Reasoning, and Using Provenance, Trust
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty
• Multiagent Systems: Multiagent Systems under Uncertainty

Application Category

• Web Mining and Content Analysis: Web data provenance, reliability, and authenticity
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Mysterious citations are routinely appearing in peer-reviewed publications throughout the scientific community. In this paper, we developed an automated pipeline and examine the proceedings of four major high-performance computing conferences, comparing the accuracy of citations between the 2021 and 2025 proceedings. While none of the 2021 papers contained mysterious citations, every 2025 proceeding did, impacting 2-6\% of published papers. In addition, we observe a sharp rise in paper title and authorship errors, motivating the need for stronger citation-verification practice. No author within our dataset acknowledged using AI to generate citations even though all four conference policies required it, indicating current policies are insufficient.

Problem

• mysterious citations
• citation accuracy
• AI-generated citations
• authorship errors
• citation verification

Innovation

• mysterious citations
• automated pipeline
• citation accuracy
• AI-generated citations
• scientific integrity