Inside CORE-KG: Evaluating Structured Prompting and Coreference Resolution for Knowledge Graphs

2025-10-30

AI Summary

Legal texts exhibit complex structures, ambiguous coreference, and fragmented information, leading to node duplication and prominent legal noise in knowledge graph (KG) construction. Method: This paper proposes a novel approach integrating type-aware coreference resolution with domain-guided structured prompting, the first to jointly embed both components into a large language model (LLM)-driven KG construction pipeline. Contribution/Results: Evaluated systematically on smuggling crime cases, ablation studies show that removing coreference resolution increases node duplication by 28.32%, while omitting structured prompting escalates noisy nodes by 73.33%. The method significantly enhances the completeness, accuracy, and domain adaptability of legal KGs. It provides an interpretable and reproducible technical framework for automated judicial text analysis.

Technology Category

• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion
• Natural Language Processing: Safety and Robustness
• Knowledge Representation and Reasoning: Knowledge Representation Languages

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Bridging structured and unstructured data

Abstract

Human smuggling networks are increasingly adaptive and difficult to analyze. Legal case documents offer critical insights but are often unstructured, lexically dense, and filled with ambiguous or shifting references, which pose significant challenges for automated knowledge graph (KG) construction. While recent LLM-based approaches improve over static templates, they still generate noisy, fragmented graphs with duplicate nodes due to the absence of guided extraction and coreference resolution. The recently proposed CORE-KG framework addresses these limitations by integrating a type-aware coreference module and domain-guided structured prompts, significantly reducing node duplication and legal noise. In this work, we present a systematic ablation study of CORE-KG to quantify the individual contributions of its two key components. Our results show that removing coreference resolution results in a 28.32% increase in node duplication and a 4.32% increase in noisy nodes, while removing structured prompts leads to a 4.34% increase in node duplication and a 73.33% increase in noisy nodes. These findings offer empirical insights for designing robust LLM-based pipelines for extracting structured representations from complex legal texts.

Problem

• Analyzing adaptive human smuggling networks from legal documents
• Reducing noise and duplication in knowledge graph construction
• Evaluating coreference resolution and structured prompting contributions

Innovation

• Integrating type-aware coreference resolution module
• Applying domain-guided structured prompting techniques
• Reducing node duplication and noise significantly