CiteEval: Principle-Driven Citation Evaluation for Source Attribution
Existing NLI-based citation evaluation methods support only coarse-grained (binary or ternary) judgments of support, failing to capture fine-grained citation quality across diverse contexts—including user queries, generated text, cited sources, and retrieval context. Method: We propose the first principle-driven, fine-grained citation evaluation framework integrating all four contextual dimensions (query–generation–source–context); construct CiteBench, the first high-quality, multi-domain, human-annotated benchmark; and design CiteEval-Auto, an automated metric suite aligned with human judgment and cognitive principles, incorporating retrieval-context modeling and multidimensional citation principles. Results: Experiments show CiteEval-Auto achieves over 35% higher correlation with human evaluations than prior NLI-based metrics, while offering superior scalability, interpretability, and fidelity.