CDFM: Towards a General-Purpose Causal Discovery Foundation Model

2026-07-13

AI Summary

This work addresses the challenges posed by the heterogeneity and ever-increasing scale of real-world data, which render traditional causal discovery methods fragmented and poorly scalable. To overcome these limitations, the paper introduces CDFM—the first general-purpose foundation model for causal discovery—that models unknown causal mechanisms as latent variables within a variational framework and leverages large-scale synthetic structural causal models for pretraining, enabling zero-shot inference of causal structures. The approach innovatively highlights the critical role of causal priors in identifiability and incorporates a modular learning architecture based on variational decomposition. Experimental results demonstrate that CDFM significantly outperforms existing algorithms across diverse unseen scenarios, exhibiting strong generalization capabilities and practical utility.

Abstract

Causal discovery, the process of recovering underlying causal structures from observational data, is a fundamental pursuit across scientific disciplines. Over the past decades, numerous algorithms have been developed to tackle this challenge through workflows tailored to the specific causal mechanisms underlying each type of dataset, demonstrating effectiveness across a wide range of applications. However, as the volume and heterogeneity of real-world data continue to grow, this dataset-specific approach inevitably leads to a fragmented, test-driven paradigm that struggles to scale to the demands of modern scientific discovery. To address this, we formulate the Causal Discovery Foundation Model (CDFM) as a unified, general-purpose framework for zero-shot structural inference. To ensure reliable generalization across unknown domains, we first investigate the theoretical boundaries of causal identifiability, revealing the indispensable role of causal prior mechanisms in this process. Building on these insights, we formulate a principled variational framework that treats unknown causal mechanisms as latent variables and mathematically decomposes the intractable marginal likelihood into distinct, tractable learning modules. The variational decomposition provides a conceptual design principle for the architecture design of CDFM, while comprehensive causal knowledge guides the large-scale synthesis of our pretraining data. By pretraining on a massive, highly diverse space of synthetic structural causal models, CDFM successfully internalizes complex statistical asymmetries. Extensive experiments demonstrate that CDFM consistently outperforms traditional algorithms, driving a paradigm shift toward a general-purpose causal discovery foundation model.

Problem

• causal discovery
• foundation model
• general-purpose
• structural inference
• data heterogeneity

Innovation

• Causal Discovery
• Foundation Model
• Variational Inference
• Structural Causal Model
• Zero-shot Learning