CiteGuard: Faithful Citation Attribution for LLMs via Retrieval-Augmented Validation

2025-10-14

AI Summary

Large language models (LLMs) often generate inaccurate citations in scientific writing and deviate from human authors’ citation practices. To address this, we propose “citation attribution alignment”—a novel paradigm that reframes citation evaluation as a task of determining whether a generated citation matches the actual citation choice a human author would make. We introduce CiteGuard, a retrieval-aware proxy evaluation framework that integrates LLM reasoning with external knowledge retrieval. Through multi-step verification, CiteGuard achieves fine-grained alignment between generated citations and human citation behavior, while also supporting identification of semantically equivalent yet syntactically distinct valid alternatives. Evaluated on the CiteME benchmark, CiteGuard achieves 65.4% accuracy—approaching human-level performance (69.7%) and surpassing the prior state-of-the-art by 12.3%. This advancement significantly improves both the reliability and interpretability of automated citation assessment.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Large Language Models (LLMs) have emerged as promising assistants for scientific writing. However, there have been concerns regarding the quality and reliability of the generated text, one of which is the citation accuracy and faithfulness. While most recent work relies on methods such as LLM-as-a-Judge, the reliability of LLM-as-a-Judge alone is also in doubt. In this work, we reframe citation evaluation as a problem of citation attribution alignment, which is assessing whether LLM-generated citations match those a human author would include for the same text. We propose CiteGuard, a retrieval-aware agent framework designed to provide more faithful grounding for citation validation. CiteGuard improves the prior baseline by 12.3%, and achieves up to 65.4% accuracy on the CiteME benchmark, on par with human-level performance (69.7%). It also enables the identification of alternative but valid citations.

Problem

• Ensuring LLM-generated citations match human author standards
• Improving citation accuracy and faithfulness in scientific writing
• Providing retrieval-augmented validation for reliable citation attribution

Innovation

• Retrieval-augmented agent framework for citation validation
• Reframes citation evaluation as attribution alignment problem
• Identifies alternative valid citations through retrieval methods