Large Language Models and Causal Inference in Collaboration: A Comprehensive Survey

2024-03-14

arXiv.org

AI Summary

This study addresses key limitations of large language models (LLMs) in prediction accuracy, fairness, robustness, and interpretability by proposing a bidirectional synergy framework integrating LLMs with causal inference. Methodologically, it unifies structural causal models (SCMs), counterfactual reasoning, intervention modeling, and causal representation learning with LLM-specific techniques—including prompt engineering, chain-of-thought reasoning, and self-consistency inference. The contributions are threefold: (1) the first systematic formulation of a bidirectional paradigm—leveraging causal methods to enhance LLMs’ reasoning, fairness, safety, and interpretability, while simultaneously exploiting LLMs to improve causal discovery and causal effect estimation; (2) comprehensive coverage across four domains: reasoning augmentation, fair and safe governance, interpretability completion, and multimodal causal modeling; and (3) introduction of a unified evaluation framework to advance trustworthy AI.

Technology Category

• Machine Learning: Causal Learning
• Reasoning under Uncertainty: Causality
• Natural Language Processing: (Large) Language Models

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Causal inference has shown potential in enhancing the predictive accuracy, fairness, robustness, and explainability of Natural Language Processing (NLP) models by capturing causal relationships among variables. The emergence of generative Large Language Models (LLMs) has significantly impacted various NLP domains, particularly through their advanced reasoning capabilities. This survey focuses on evaluating and improving LLMs from a causal view in the following areas: understanding and improving the LLMs' reasoning capacity, addressing fairness and safety issues in LLMs, complementing LLMs with explanations, and handling multimodality. Meanwhile, LLMs' strong reasoning capacities can in turn contribute to the field of causal inference by aiding causal relationship discovery and causal effect estimations. This review explores the interplay between causal inference frameworks and LLMs from both perspectives, emphasizing their collective potential to further the development of more advanced and equitable artificial intelligence systems.

Problem

• Enhancing NLP models via causal inference
• Improving LLMs' reasoning and fairness
• Exploring LLMs and causal inference interplay

Innovation

• LLMs enhance causal inference accuracy
• Causal frameworks improve LLMs' reasoning
• Mutual benefits between LLMs and causality