Analysis of Broken Randomized Experiments by Principal Stratification

2024-05-27

AI Summary

This paper addresses bias in causal effect estimation arising from “broken” randomized controlled trials—specifically, due to noncompliance, truncation by death, and missing data. It develops a principal stratification–based causal framework targeting the principal stratum effect (PSE) among compliers and the always-survivors effect (ASE) among immortal subjects. Methodologically, it proposes a covariate-adjusted interventional estimator that weakens conventional identification assumptions—namely, monotonicity and exclusion restrictions—while preserving identifiability, and establishes its large-sample asymptotic properties. Applied to the Job Corps employment training study, the method yields statistically significant estimates: training substantially improves long-term employment rates and earnings. Empirical results demonstrate the framework’s robustness and practical utility in realistic, complex trial settings characterized by multiple sources of interference and outcome incompleteness.

Technology Category

• Machine Learning: Causal Learning
• Reasoning under Uncertainty: Causality
• Intelligent Robots: State Estimation

Application Category

• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Security and Privacy: Large-scale security measurements
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

Although randomized controlled trials have long been regarded as the ``gold standard'' for evaluating treatment effects, there is no natural prevention from post-treatment events. For example, non-compliance makes the actual treatment different from the assigned treatment, truncation-by-death renders the outcome undefined or ill-defined, and missingness prevents the outcomes from being measured. In this paper, we develop a statistical analysis framework using principal stratification to investigate the treatment effect in broken randomized experiments. The average treatment effect in compliers and always-survivors is adopted as the target causal estimand. We establish the asymptotic property for the estimator. To relax the identification assumptions, we also propose an interventionist estimand defined in compliers by adjusting for baseline covariates. We apply the framework to study the effect of training on earnings in the Job Corps study and find that the training program improves employment and earnings in the long term.

Problem

• Analyzing treatment effects in broken randomized experiments
• Addressing non-compliance and truncation-by-death issues
• Estimating causal effects using principal stratification

Innovation

• Principal stratification for broken randomized experiments
• Average treatment effect in compliers and always-survivors
• Interventionist estimand with baseline covariate adjustment