Synthetic Control Misconceptions: Recommendations for Practice

2026-03-19

AI Summary

This study addresses widespread misconceptions in the practical application of the synthetic control method, particularly concerning its reliance on covariates, claims of robustness, and prevailing model selection criteria—assertions often lacking empirical validation and potentially undermining causal inference reliability. Through rigorous theoretical analysis and extensive simulation experiments, the paper systematically evaluates these common misunderstandings and compares the performance of standard implementations against alternative approaches. The findings uncover critical pitfalls in current practices and, grounded in empirical evidence, offer concrete recommendations for more principled implementation and interpretation. By doing so, the work provides researchers with a practical guide to significantly enhance the quality and credibility of causal inferences derived from synthetic control methods.

Technology Category

• Reasoning under Uncertainty: Causality
• Game Theory and Economic Paradigms: Imperfect Information
• Machine Learning: Causal Learning

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

To estimate the causal effect of an intervention, researchers need to identify a control group that represents what might have happened to the treatment group in the absence of that intervention. This is challenging without a randomized experiment and further complicated when few units (possibly only one) are treated. Nevertheless, when data are available on units over time, synthetic control (SC) methods provide an opportunity to construct a valid comparison by differentially weighting control units that did not receive the treatment so that their resulting pre-treatment trajectory is similar to that of the treated unit. The hope is that this weighted ``pseudo-counterfactual" can serve as a valid counterfactual in the post-treatment time period. Since its origin twenty years ago, SC has been used over 5,000 times in the literature (Web of Science, December 2025), leading to a proliferation of descriptions of the method and guidance on proper usage that is not always accurate and does not always align with what the original developers appear to have intended. As such, a number of accepted pieces of wisdom have arisen: (1) SC is robust to various implementations; (2) covariates are unnecessary, and (3) pre-treatment prediction error should guide model selection. We describe each in detail and conduct simulations that suggest, both for standard and alternative implementations of SC, that these purported truths are not supported by empirical evidence and thus actually represent misconceptions about best practice. Instead of relying on these misconceptions, we offer practical advice for more cautious implementation and interpretation of results.

Problem

• synthetic control
• causal inference
• misconceptions
• counterfactual
• model selection

Innovation

• synthetic control
• causal inference
• misconceptions
• model selection
• simulation study