Integrating Large Language Models in Causal Discovery: A Statistical Causal Approach

2024-02-02

arXiv.org

AI Summary

In statistical causal discovery (SCD), the systematic incorporation of domain expert knowledge remains challenging, often leading to inconsistent causal models. Method: This paper proposes the Statistical Causal Prompting (SCP) framework—the first to deeply integrate large language models (LLMs) with classical causal discovery algorithms. SCP employs a structured prompting mechanism for knowledge-guided causal inference, enabling zero-shot domain transfer and closed-loop expert validation; it further injects constraints and augments prior knowledge to transform LLM-extracted semantic knowledge into verifiable causal constraints. Contribution/Results: Experiments demonstrate significant improvements in causal graph accuracy on synthetic data and multiple unseen real-world datasets. SCP effectively mitigates data bias while preserving interpretability and theoretical grounding. The implementation is publicly available as open-source software.

Technology Category

• Machine Learning: Causal Learning
• Reasoning under Uncertainty: Causality
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

In practical statistical causal discovery (SCD), embedding domain expert knowledge as constraints into the algorithm is important for creating consistent, meaningful causal models, despite the challenges in the systematic acquisition of background knowledge. To overcome these challenges, this paper proposes a novel method for causal inference, in which SCD and knowledge based causal inference (KBCI) with a large language model (LLM) are synthesized through ``statistical causal prompting (SCP)'' for LLMs and prior knowledge augmentation for SCD. Experiments have revealed that the results of LLM-KBCI and SCD augmented with LLM-KBCI approach the ground truths, more than the SCD result without prior knowledge. It has also been revealed that the SCD result can be further improved if the LLM undergoes SCP. Furthermore, with an unpublished real-world dataset, we have demonstrated that the background knowledge provided by the LLM can improve the SCD on this dataset, even if this dataset has never been included in the training data of the LLM. For future practical application of this proposed method across important domains such as healthcare, we also thoroughly discuss the limitations, risks of critical errors, expected improvement of techniques around LLMs, and realistic integration of expert checks of the results into this automatic process, with SCP simulations under various conditions both in successful and failure scenarios. The careful and appropriate application of the proposed approach in this work, with improvement and customization for each domain, can thus address challenges such as dataset biases and limitations, illustrating the potential of LLMs to improve data-driven causal inference across diverse scientific domains. The code used in this work is publicly available at: www.github.com/mas-takayama/LLM-and-SCD

Problem

• Statistical Causal Discovery
• Expert Knowledge Integration
• Causal Modeling

Innovation

• Statistical Causal Discovery
• Large Language Model Integration
• Knowledge-based Causal Inference