Smooth Multi-Policy Causal Effect Estimation in Longitudinal Settings

2026-05-13

AI Summary

This work addresses a key limitation of conventional longitudinal causal inference methods, which estimate multiple dynamic treatment strategies in isolation and thus fail to share information across counterfactuals, leading to increased second-order bias and finite-sample variance. To overcome this, the authors propose a policy-aware reparameterization framework for incremental conditional expectation (ICE) Q-functions, termed PEQ-Net. This framework employs a shared policy encoder and kernel mean embeddings to construct a representation space that captures policy discrepancies, thereby structurally controlling second-order remainder bias. Variance stability is further enhanced through an LTMLE correction step. Empirical evaluation on semi-synthetic data demonstrates that PEQ-Net substantially outperforms existing ICE approaches, particularly when evaluating similar policies, achieving markedly lower root mean squared error.

Abstract

Comparative evaluation of multiple dynamic treatment policies is essential for healthcare and policy decisions, yet conventional longitudinal causal inference methods estimate each in isolation, preventing information sharing across counterfactuals. We demonstrate that this separate estimation paradigm induces a structurally uncontrolled second-order bias, inflating finite-sample variance even after standard debiasing with longitudinal targeted maximum likelihood estimation(LTMLE). To address this, we propose a policy-aware reparameterization of Iterative Conditional Expectation (ICE) Q-functions that enables joint estimation through shared representations. We implement this approach in the Policy-Encoded Q Network (PEQ-Net), an architecture centered on a shared policy encoder. The encoder is trained using kernel mean embeddings, ensuring that the learned representation space reflects population-level policy dissimilarities. After applying an LTMLE correction step, we prove this design imposes a structural constraint on the second-order remainder, thereby stabilizing finite-sample variance. Experiments on semi-synthetic datasets demonstrate that PEQ-Net consistently outperforms existing ICE-based methods, achieving substantial reductions in root-mean-square error, particularly when evaluating closely related policies.

Problem

• longitudinal causal inference
• dynamic treatment policies
• counterfactual estimation
• finite-sample variance
• second-order bias

Innovation

• policy-aware reparameterization
• shared policy encoder
• kernel mean embeddings
• longitudinal targeted maximum likelihood estimation
• joint causal effect estimation