C$^2$-Cite: Contextual-Aware Citation Generation for Attributed Large Language Models

2025-11-19

AI Summary

Existing attributable large language models struggle to accurately interpret the contextual semantics of citation markers (e.g., [i]), often resulting in misaligned references and insufficient knowledge integration. To address this, this work proposes the C²-Cite framework, which explicitly models the semantic relationship between citation markers and their referenced content for the first time. By incorporating a context-aware citation alignment mechanism—embedding document context into citation representations and introducing a citation routing function for precise decoding of marker indices—the approach transforms static citation placeholders into dynamic knowledge pointers. Evaluated on three datasets from the ALCE benchmark, C²-Cite++ achieves an average improvement of 5.8% in citation quality and a 17.4% gain in answer accuracy, significantly outperforming current state-of-the-art methods.

Abstract

The attribution technique enhances the credibility of LLMs by adding citations to the generated sentences, enabling users to trace back to the original sources and verify the reliability of the output. However, existing instruction-tuned attributed LLMs often fail to properly interpret the contextual semantics of citation symbols (e.g., [i]) during text generation. This shortcoming arises from their insufficient awareness of the context information surrounding citation markers, which in turn leads to disjointed references and poor integration of retrieved knowledge into the generated content. To address this issue, we propose a novel \textbf{C}ontextual-aware \textbf{C}itation generation framework (\textbf{C$^2$}-\textbf{Cite}) that explicitly integrates the semantic relationships between citation markers and their referenced content. Specifically, a contextual citation alignment mechanism is adopted: it first encodes the retrieved document contexts into the symbol representation of citations, then aligns the marker numbers by decoding information from a citation router function. This mechanism enables the transformation of citation markers from generic placeholders into active knowledge pointers that link to the referenced source information. Experimental results on the ALCE benchmark across three datasets validate our framework C$^2$-Cite++: it outperforms the SOTA baseline by an average of 5.8\% in citation quality and 17.4\% in response correctness. The implementation is publicly available at https://github.com/BAI-LAB/c2cite

Problem

• citation generation
• attributed LLMs
• contextual awareness
• knowledge integration
• semantic alignment

Innovation

• contextual-aware citation
• attributed LLMs
• citation alignment
• knowledge integration
• citation generation