Citation Parsing and Analysis with Language Models

2025-05-21

AI Summary

Knowledge produced in the Global South has been systematically underrepresented in mainstream citation indexes, reinforcing academic colonial structures and marginalizing scholars from these regions. Method: This study develops the first open-source citation parsing tool designed for monitoring global knowledge flows. We propose a sequence-labeling and instruction-tuning framework leveraging lightweight open-source LLMs—including Qwen3 and Phi-3—and conduct the first systematic evaluation of zero-shot structured citation parsing. We further introduce a novel paired citation dataset covering diverse publication statuses, comprising raw text and fully structured annotations. Contribution: With minimal fine-tuning, Qwen3-0.6B achieves high accuracy across all citation fields in zero-shot settings, surpassing current state-of-the-art methods. End-to-end parsing is completed within 32 inference steps, drastically lowering deployment barriers. The framework enables scalable, decentralized infrastructure for enhancing visibility of Southern Hemisphere scholarship—particularly suitable for resource-constrained environments.

Technology Category

• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion
• Humans and AI: Crowd Sourcing and Human Computation
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)

Application Category

• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

A key type of resource needed to address global inequalities in knowledge production and dissemination is a tool that can support journals in understanding how knowledge circulates. The absence of such a tool has resulted in comparatively less information about networks of knowledge sharing in the Global South. In turn, this gap authorizes the exclusion of researchers and scholars from the South in indexing services, reinforcing colonial arrangements that de-center and minoritize those scholars. In order to support citation network tracking on a global scale, we investigate the capacity of open-weight language models to mark up manuscript citations in an indexable format. We assembled a dataset of matched plaintext and annotated citations from preprints and published research papers. Then, we evaluated a number of open-weight language models on the annotation task. We find that, even out of the box, today's language models achieve high levels of accuracy on identifying the constituent components of each citation, outperforming state-of-the-art methods. Moreover, the smallest model we evaluated, Qwen3-0.6B, can parse all fields with high accuracy in $2^5$ passes, suggesting that post-training is likely to be effective in producing small, robust citation parsing models. Such a tool could greatly improve the fidelity of citation networks and thus meaningfully improve research indexing and discovery, as well as further metascientific research.

Problem

• Develops tool to analyze global knowledge circulation disparities
• Enhances citation network tracking with open-weight language models
• Improves research indexing accuracy for Global South scholars

Innovation

• Uses open-weight language models for citation parsing
• Evaluates models on annotated citation dataset
• Small model Qwen3-0.6B achieves high accuracy