HalluCitation Matters: Revealing the Impact of Hallucinated References with 300 Hallucinated Papers in ACL Conferences

2026-01-26

AI Summary

This study addresses the problem of “HalluCitation”—the citation of non-existent references—in academic publications by conducting the first large-scale empirical analysis of all papers accepted to the three premier computational linguistics conferences (ACL, NAACL, and EMNLP) in 2024–2025. Leveraging bibliographic comparison, citation validation, and text mining techniques, the authors systematically verify the authenticity of cited references. Their analysis reveals that nearly 300 papers contain at least one HalluCitation, with over 100 such instances originating from the main conference and Findings of EMNLP 2025, indicating a rapidly growing prevalence of this issue in the field. This work not only introduces the term “HalluCitation” to quantify its impact but also provides the first evidence of its widespread occurrence in top-tier venues, offering critical insights for enhancing scholarly integrity.

Technology Category

• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Web Mining and Content Analysis: Web data provenance, reliability, and authenticity
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Recently, we have often observed hallucinated citations or references that do not correspond to any existing work in papers under review, preprints, or published papers. Such hallucinated citations pose a serious concern to scientific reliability. When they appear in accepted papers, they may also negatively affect the credibility of conferences. In this study, we refer to hallucinated citations as"HalluCitation"and systematically investigate their prevalence and impact. We analyze all papers published at ACL, NAACL, and EMNLP in 2024 and 2025, including main conference, Findings, and workshop papers. Our analysis reveals that nearly 300 papers contain at least one HalluCitation, most of which were published in 2025. Notably, half of these papers were identified at EMNLP 2025, the most recent conference, indicating that this issue is rapidly increasing. Moreover, more than 100 such papers were accepted as main conference and Findings papers at EMNLP 2025, affecting the credibility.

Problem

• Hallucinated citations
• scientific reliability
• academic credibility
• ACL conferences
• reference integrity

Innovation

• Hallucinated citations
• scientific reliability
• ACL conferences
• EMNLP
• academic integrity