ACTIVA: Amortized Causal Effect Estimation without Graphs via Transformer-based Variational Autoencoder

2025-03-03

AI Summary

This paper addresses the problem of predicting the full distribution of intervention effects in settings where the causal graph is unknown. We propose a nonparametric, amortized causal inference framework that requires no prior knowledge of the causal structure. Methodologically, the approach integrates a conditional variational autoencoder (CVAE) with a causal Transformer encoder, taking observed data and the target intervention as inputs, and outputs the complete post-intervention outcome distribution via a Gaussian mixture model; it further supports zero-shot transfer to unseen datasets. Unlike conventional methods reliant on known causal graphs or strong parametric assumptions, our framework significantly improves generalizability and practical applicability. Experiments on synthetic and semi-synthetic benchmarks demonstrate high-accuracy intervention distribution prediction. To our knowledge, this is the first method enabling graph-agnostic, amortized, end-to-end distributional causal effect estimation.

Technology Category

• Machine Learning: Causal Learning
• Reasoning under Uncertainty: Causality
• Computer Vision: Diffusion Models for Vision

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Predicting the distribution of outcomes under hypothetical interventions is crucial in domains like healthcare, economics, and policy-making. Current methods often rely on strong assumptions, such as known causal graphs or parametric models, and lack amortization across problem instances, limiting their practicality. We propose a novel transformer-based conditional variational autoencoder architecture, named ACTIVA, that extends causal transformer encoders to predict causal effects as mixtures of Gaussians. Our method requires no causal graph and predicts interventional distributions given only observational data and a queried intervention. By amortizing over many simulated instances, it enables zero-shot generalization to novel datasets without retraining. Experiments demonstrate accurate predictions for synthetic and semi-synthetic data, showcasing the effectiveness of our graph-free, amortized causal inference approach.

Problem

• Estimates causal effects without requiring known causal graphs
• Predicts interventional distributions using only observational data
• Enables zero-shot generalization to new datasets without retraining

Innovation

• Transformer-based variational autoencoder for causal effect estimation
• Predicts interventional distributions without causal graphs
• Enables zero-shot generalization to novel datasets