Time After Time: Deep-Q Effect Estimation for Interventions on When and What to do

2025-03-20

AI Summary

Estimating irregular-time causal effects—jointly determining *when* and *what* to intervene—is challenging in domains such as healthcare, robotics, and finance, where events occur at continuous, non-uniform timestamps and timing itself is a causal factor. Method: We propose Earliest Disagreement Q-Evaluation (EDQ), the first method to jointly model intervention timing and action selection under continuous, irregular time. EDQ leverages a recursive Q-function formulation, enabling unbiased causal effect estimation without time discretization or neglecting timing effects, and natively supports sequence models (e.g., Transformers). Contribution/Results: We establish theoretical consistency of EDQ under mild assumptions. Empirically, EDQ achieves significant improvements in estimation accuracy over baselines on survival analysis and tumor growth simulation tasks, demonstrating both statistical validity and practical utility for real-world irregular-time decision-making.

Technology Category

• Reasoning under Uncertainty: Sequential Decision Making
• Machine Learning: Causal Learning
• Knowledge Representation and Reasoning: Action, Change, and Causality

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Problems in fields such as healthcare, robotics, and finance requires reasoning about the value both of what decision or action to take and when to take it. The prevailing hope is that artificial intelligence will support such decisions by estimating the causal effect of policies such as how to treat patients or how to allocate resources over time. However, existing methods for estimating the effect of a policy struggle with emph{irregular time}. They either discretize time, or disregard the effect of timing policies. We present a new deep-Q algorithm that estimates the effect of both when and what to do called Earliest Disagreement Q-Evaluation (EDQ). EDQ makes use of recursion for the Q-function that is compatible with flexible sequence models, such as transformers. EDQ provides accurate estimates under standard assumptions. We validate the approach through experiments on survival time and tumor growth tasks.

Problem

• Estimates causal effects of timing and action decisions.
• Addresses irregular time challenges in policy effect estimation.
• Introduces EDQ for accurate when and what decision evaluations.

Innovation

• Deep-Q algorithm for timing and action decisions
• Recursive Q-function compatible with transformers
• Accurate effect estimation under standard assumptions