Amortized Causal Discovery with Prior-Fitted Networks

2025-12-03

AI Summary

Existing differentiable penalized likelihood methods suffer from biased likelihood estimation as sample size increases, leading to failure in causal structure recovery. This paper introduces the first amortized causal discovery framework based on Prior-Fitting Networks (PFNs), which abandons sample-dependent, gradient-based direct likelihood modeling and instead employs PFNs to achieve high-accuracy, data-adaptive amortized likelihood estimation. The method significantly improves structural robustness under small-to-moderate sample sizes. On synthetic, simulated, and real-world benchmarks, it achieves an average 12–28% higher structural recovery accuracy than state-of-the-art baselines—including NOTEARS and DAG-GNN—while reducing PFN-based likelihood estimation error by up to 41%. This work constitutes the first application of PFNs to causal discovery and empirically validates their effectiveness and superiority in amortized causal structure learning.

Technology Category

• Machine Learning: Causal Learning
• Reasoning under Uncertainty: Relational Probabilistic Models
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

In recent years, differentiable penalized likelihood methods have gained popularity, optimizing the causal structure by maximizing its likelihood with respect to the data. However, recent research has shown that errors in likelihood estimation, even on relatively large sample sizes, disallow the discovery of proper structures. We propose a new approach to amortized causal discovery that addresses the limitations of likelihood estimator accuracy. Our method leverages Prior-Fitted Networks (PFNs) to amortize data-dependent likelihood estimation, yielding more reliable scores for structure learning. Experiments on synthetic, simulated, and real-world datasets show significant gains in structure recovery compared to standard baselines. Furthermore, we demonstrate directly that PFNs provide more accurate likelihood estimates than conventional neural network-based approaches.

Problem

• Addresses limitations of likelihood estimator accuracy in causal discovery
• Uses Prior-Fitted Networks to amortize data-dependent likelihood estimation
• Improves structure recovery across synthetic and real-world datasets

Innovation

• Uses Prior-Fitted Networks to amortize likelihood estimation
• Improves structure recovery accuracy over standard baselines
• Provides more reliable scores for causal discovery