Enhancing Natural Language Inference Performance with Knowledge Graph for COVID-19 Automated Fact-Checking in Indonesian Language

2024-08-22

arXiv.org

AI Summary

To address the performance saturation of Natural Language Inference (NLI) models in Indonesian COVID-19 fact-checking—stemming from domain-specific knowledge gaps—this paper proposes the first end-to-end NLI framework integrating an Indonesian-language COVID-19 knowledge graph (KG Bahasa Indonesia). The framework comprises three modules: a fact module, an NLI module, and a classification module. It jointly encodes textual semantics via BERT, injects structured knowledge using TransE embeddings, and enables knowledge-semantic协同 reasoning through vector concatenation and an MLP classifier. Evaluated on a newly constructed Indonesian fact-checking dataset, the model achieves 86.16% accuracy, substantially outperforming text-only baselines. This work constitutes the first explicit integration of a domain-specific knowledge graph into NLI for a low-resource language, empirically demonstrating that external knowledge significantly enhances the robustness of premise-hypothesis entailment judgments.

Technology Category

• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Knowledge Representation and Reasoning: Knowledge Representation Languages
• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Automated fact-checking is a key strategy to overcome the spread of COVID-19 misinformation on the internet. These systems typically leverage deep learning approaches through Natural Language Inference (NLI) to verify the truthfulness of information based on supporting evidence. However, one challenge that arises in deep learning is performance stagnation due to a lack of knowledge during training. This study proposes using a Knowledge Graph (KG) as external knowledge to enhance NLI performance for automated COVID-19 fact-checking in the Indonesian language. The proposed model architecture comprises three modules: a fact module, an NLI module, and a classifier module. The fact module processes information from the KG, while the NLI module handles semantic relationships between the given premise and hypothesis. The representation vectors from both modules are concatenated and fed into the classifier module to produce the final result. The model was trained using the generated Indonesian COVID-19 fact-checking dataset and the COVID-19 KG Bahasa Indonesia. Our study demonstrates that incorporating KGs can significantly improve NLI performance in fact-checking, achieving the best accuracy of 0,8616. This suggests that KGs are a valuable component for enhancing NLI performance in automated fact-checking.

Problem

• Enhancing NLI performance for COVID-19 fact-checking in Indonesian
• Overcoming knowledge gaps in deep learning for misinformation detection
• Integrating Knowledge Graphs to improve automated fact-checking accuracy

Innovation

• Knowledge Graph enhances NLI for fact-checking
• Three-module architecture processes KG and NLI
• Model achieves high accuracy in Indonesian COVID-19 verification