Overlap-weighted orthogonal meta-learner for treatment effect estimation over time

2025-10-22

AI Summary

Estimating heterogeneous treatment effects (HTE) in time-varying environments suffers from severe overlap deficiency: the probability of observing long intervention sequences decays exponentially, causing explosive variance in conventional meta-learners. To address this, we propose an overlap-weighted orthogonal meta-learning framework that enhances estimation stability by focusing inference on high-probability intervention sequence regions. Our core contribution is a novel overlap-weighted Neyman-orthogonal risk function—robust to nuisance function misspecification and fully model-agnostic—enabling seamless integration with expressive sequence models such as Transformers and LSTMs. Extensive experiments across diverse time-varying causal settings demonstrate that our method substantially reduces estimation variance and improves both accuracy and robustness of HTE inference. This work establishes a new paradigm for modeling long-horizon heterogeneous causal effects under dynamic interventions.

Technology Category

• Machine Learning: Causal Learning
• Search and Optimization: Metareasoning and Metaheuristics
• Reasoning under Uncertainty: Causality

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Estimating heterogeneous treatment effects (HTEs) in time-varying settings is particularly challenging, as the probability of observing certain treatment sequences decreases exponentially with longer prediction horizons. Thus, the observed data contain little support for many plausible treatment sequences, which creates severe overlap problems. Existing meta-learners for the time-varying setting typically assume adequate treatment overlap, and thus suffer from exploding estimation variance when the overlap is low. To address this problem, we introduce a novel overlap-weighted orthogonal (WO) meta-learner for estimating HTEs that targets regions in the observed data with high probability of receiving the interventional treatment sequences. This offers a fully data-driven approach through which our WO-learner can counteract instabilities as in existing meta-learners and thus obtain more reliable HTE estimates. Methodologically, we develop a novel Neyman-orthogonal population risk function that minimizes the overlap-weighted oracle risk. We show that our WO-learner has the favorable property of Neyman-orthogonality, meaning that it is robust against misspecification in the nuisance functions. Further, our WO-learner is fully model-agnostic and can be applied to any machine learning model. Through extensive experiments with both transformer and LSTM backbones, we demonstrate the benefits of our novel WO-learner.

Problem

• Estimating heterogeneous treatment effects with severe overlap issues
• Addressing exploding variance in time-varying treatment sequences
• Providing robust estimation against nuisance function misspecification

Innovation

• Overlap-weighted orthogonal meta-learner for treatment effect estimation
• Neyman-orthogonal population risk function minimizing weighted oracle risk
• Model-agnostic framework applicable to transformer and LSTM backbones