Using LLMs and ontologies to extract causal relationships from medical abstracts

2025-02-16

Procedia Computer Science

AI Summary

To address the unreliability and non-standardization of outputs generated by large language models (LLMs) in biomedical causal relation extraction, this paper proposes an ontology-enhanced LLM collaborative modeling framework. Our method uniquely integrates the semantic understanding capability of an open-source LLM (Llama-3) with the formal logical constraints encoded in the Unified Medical Language System (UMLS) ontology, augmented by causal pattern–guided prompt engineering and rule-driven post-hoc reasoning. This enables precise identification and standardized representation of causal triples from unstructured biomedical abstracts. The approach effectively mitigates challenges posed by ambiguity, implicitness, and terminological heterogeneity in biomedical causal expressions. Evaluated on the MedCausa benchmark, our method achieves an F1 score of 82.4%—an 11.7-point improvement over a vanilla LLM baseline—and attains a 93.2% standardization coverage rate for extracted causal triples, significantly enhancing result verifiability and domain-specific adaptability.

Technology Category

• Reasoning under Uncertainty: Causality
• Machine Learning: Causal Learning
• Knowledge Representation and Reasoning: Action, Change, and Causality

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Problem

• Reduces hallucinations in LLM outputs
• Enhances factual precision in responses
• Improves clarity of chatbot-generated answers

Innovation

• Knowledge Graph integration
• Retrieval-Augmented Generation technique
• Vector-based retrieval process