CasualSynth: Generating Structurally Sound Synthetic Data

2026-05-17

AI Summary

This work addresses the challenge that synthetic data generated by large language models often violates the causal structure of the target domain due to interference from pretraining priors. To resolve this, the authors propose a three-stage decoupling framework: first, a causal skeleton satisfying the global Markov property is generated using a structural causal model; second, a large language model acts as a constrained realization module to map this skeleton into high-dimensional semantic observations; and third, an iterative consistency verification module detects and corrects structural violations, forming a closed-loop optimization. This approach is the first to decouple causal structure generation from semantic realization, incorporates iterative validation to mitigate semantic backdoor issues, and enables principled generation of interventional and counterfactual data. On the ASIA, ALARM, and MIMIC-Struct benchmarks, the method achieves a false positive rate in conditional independence tests close to the nominal α = 0.05 and exceeds 96% semantic realizability even with a 70B-parameter model.

Abstract

Large Language Models (LLMs) generate realistic synthetic data but offer no guarantee that their outputs respect the causal mechanisms governing the target domain. We introduce CausalSynth, a framework that decouples causal structure generation from semantic realization, yielding synthetic data that is both causally valid and linguistically rich. The framework operates in three phases. First, a Structural Causal Model (SCM) - a tuple of structural equations defined over a directed acyclic graph (DAG) generates causal skeletons, i.e., variable assignments that satisfy the Global Markov Property of the governing DAG, via ancestral sampling. Second, an LLM acts as a constrained \emph{realizer}, a conditional translator that maps each skeleton to a high-dimensional observation such as a clinical note or a transaction log. Third, an Iterative Consistency Verification module detects structural violations through deterministic extraction and feeds targeted corrections back to the LLM, forming a closed-loop refinement process. We identify the Semantic Backdoor problem the systematic tendency of LLMs to override imposed causal facts with pre-training priors -- and prove that our iterative mechanism reduces the resulting selection bias relative to standard rejection sampling. On three causal benchmarks (ASIA, ALARM, and MIMIC-Struct), CausalSynth preserved conditional independencies with false-positive rates near the nominal $α=0.05$ level and achieved realizability rates above 96% with 70B-parameter LLM backbones. The framework additionally supports principled interventional and counterfactual generation through noise retention and graph mutilation.

Problem

• causal validity
• synthetic data generation
• large language models
• causal structure
• semantic backdoor

Innovation

• CausalSynth
• Structural Causal Model
• Semantic Backdoor
• Iterative Consistency Verification
• Counterfactual Generation