TextMine: LLM-Powered Knowledge Extraction for Humanitarian Mine Action

2025-09-18

AI Summary

Humanitarian demining operations generate vast volumes of unstructured field reports rich in practical knowledge, yet these remain difficult to systematically exploit. To address this, we propose an ontology-guided prompt alignment framework—the first to construct a domain-specific demining ontology and a real-world report dataset—and integrate document chunking with domain-aware prompting to enable high-quality, large language model (LLM)-driven extraction of knowledge triples. Our method innovatively employs a dual evaluation strategy combining LLM-as-a-Judge and reference-based metrics. Experiments demonstrate significant improvements over baseline approaches: a 44.2% increase in extraction accuracy, a 22.5% reduction in hallucination rate, and a 20.9% gain in structural compliance. This work establishes a reusable methodology and foundational infrastructure for building global demining knowledge graphs and supporting intelligent decision-making in humanitarian mine action.

Technology Category

• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion
• Natural Language Processing: Information Extraction
• Knowledge Representation and Reasoning: Ontologies

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Web Mining and Content Analysis: Bridging structured and unstructured data
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Humanitarian Mine Action has generated extensive best-practice knowledge, but much remains locked in unstructured reports. We introduce TextMine, an ontology-guided pipeline that uses Large Language Models to extract knowledge triples from HMA texts. TextMine integrates document chunking, domain-aware prompting, triple extraction, and both reference-based and LLM-as-a-Judge evaluation. We also create the first HMA ontology and a curated dataset of real-world demining reports. Experiments show ontology-aligned prompts boost extraction accuracy by 44.2%, cut hallucinations by 22.5%, and improve format conformance by 20.9% over baselines. While validated on Cambodian reports, TextMine can adapt to global demining efforts or other domains, transforming unstructured data into structured knowledge.

Problem

• Extracting structured knowledge from unstructured humanitarian mine action reports
• Reducing information extraction hallucinations and improving format accuracy
• Creating an adaptable ontology-guided pipeline for global demining knowledge transformation

Innovation

• Ontology-guided LLM pipeline extracts knowledge triples
• Domain-aware prompting boosts accuracy by 44.2%
• Integrated chunking, extraction and dual evaluation methods