Technical Report: Facilitating the Adoption of Causal Inference Methods Through LLM-Empowered Co-Pilot

2025-08-14

AI Summary

Estimating treatment effects (TE) from observational data remains challenging due to reliance on strong causal assumptions and complex modeling, limiting accessibility for non-experts. This paper introduces CATE-B: the first end-to-end causal analysis framework powered by LLM-based agents, integrating causal discovery, identification of the minimal uncertainty-adjustment set, structural causal modeling, uncertainty quantification, and adaptive regression model selection. Its key contribution is the formalization of the *minimal uncertainty-adjustment set*—a novel criterion balancing confounding control and estimation variance—and the use of LLMs to assist in causal graph edge orientation and structural inference, substantially lowering the barrier to entry. The system is open-sourced alongside a benchmark suite featuring diverse domains and intricate causal structures. Experiments demonstrate that CATE-B significantly outperforms state-of-the-art methods in both TE estimation accuracy and user success rate, advancing both accessibility and reliability in causal inference.

Technology Category

• Reasoning under Uncertainty: Causality
• Machine Learning: Causal Learning
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Estimating treatment effects (TE) from observational data is a critical yet complex task in many fields, from healthcare and economics to public policy. While recent advances in machine learning and causal inference have produced powerful estimation techniques, their adoption remains limited due to the need for deep expertise in causal assumptions, adjustment strategies, and model selection. In this paper, we introduce CATE-B, an open-source co-pilot system that uses large language models (LLMs) within an agentic framework to guide users through the end-to-end process of treatment effect estimation. CATE-B assists in (i) constructing a structural causal model via causal discovery and LLM-based edge orientation, (ii) identifying robust adjustment sets through a novel Minimal Uncertainty Adjustment Set criterion, and (iii) selecting appropriate regression methods tailored to the causal structure and dataset characteristics. To encourage reproducibility and evaluation, we release a suite of benchmark tasks spanning diverse domains and causal complexities. By combining causal inference with intelligent, interactive assistance, CATE-B lowers the barrier to rigorous causal analysis and lays the foundation for a new class of benchmarks in automated treatment effect estimation.

Problem

• Lowering barriers to causal inference adoption for non-experts
• Guiding users through end-to-end treatment effect estimation
• Simplifying complex causal assumptions and model selection

Innovation

• LLM-based causal model construction
• Minimal Uncertainty Adjustment Set criterion
• Tailored regression method selection