Testing Our Foundations: Citation Trends, Errors, and Emerging Hallucinations in the Computing Education Literature

2026-09-14

AI Summary

该研究针对计算教育文献中出现的虚假引用问题，通过分析ACM数字图书馆数据，识别并分类了这些错误引用，揭示了其增长趋势及对学术诚信的风险。

Abstract

Accurate references are foundational to scholarly work, enabling verification, attribution, and systematic review. However, the rapid adoption of large language models has introduced a serious integrity concern: plausible-looking but fabricated citations. Although hallucinated references are widely discussed, their visibility within specific research communities remains unclear. We address this gap by examining reference integrity at key computing education venues using ACM Digital Library data. We analyze referencing trends across 24,751 computing education papers and compare them with the broader ACM corpus of more than 723,000 papers and 15 million references. We then examine reference lists from these venues, classify common bibliographic errors, and manually identify LLM-generated hallucinations containing verifiably false information, including impossible page ranges, invented titles, and misattributed authors. In 2025, hallucinated references appeared across five SIGCSE-sponsored or in-cooperation venues. At the Technical Symposium alone, verified hallucinated references increased from 3 in 2025 to 17 in 2026, appearing in 2.3\% of 2026 proceedings papers. Although still relatively rare for now, this growth poses an integrity risk our community should not ignore.

Problem

• citation accuracy
• large language models
• hallucinations
• academic integrity
• computing education

Innovation

• Large Language Models
• Hallucinated References
• Bibliographic Errors
• Citation Integrity
• Computing Education