Statistical Decision Theory with Counterfactual Loss

2025-05-13

AI Summary

Classical statistical decision theory evaluates interventions solely based on observed outcomes, ignoring counterfactuals, and thus cannot comparatively assess the quality of feasible alternative decisions. Method: This paper integrates counterfactual loss functions into decision theory, establishing an evaluation framework that accounts for all potential outcomes. It introduces and characterizes the necessary and sufficient conditions for identifiability of counterfactual risk: strong indistinguishability and additivity of the loss function in potential outcomes. Contribution/Results: We prove that counterfactual risk is identifiable if and only if the loss is additive in potential outcomes. Moreover, when there are at least three treatment options, the counterfactual-optimal policy strictly dominates classical (observed-outcome–based) policies. This work extends the foundational theory of decision-making under the potential outcomes framework and establishes a new, identifiable, and comparable paradigm for risk assessment in causal decision-making.

Technology Category

• Reasoning under Uncertainty: Decision/Utility Theory
• Game Theory and Economic Paradigms: Adversarial Learning
• Machine Learning: Causal Learning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Classical statistical decision theory evaluates treatment choices based solely on observed outcomes. However, by ignoring counterfactual outcomes, it cannot assess the quality of decisions relative to feasible alternatives. For example, the quality of a physician's decision may depend not only on patient survival, but also on whether a less invasive treatment could have produced a similar result. To address this limitation, we extend standard decision theory to incorporate counterfactual losses--criteria that evaluate decisions using all potential outcomes. The central challenge in this generalization is identification: because only one potential outcome is observed for each unit, the associated risk under a counterfactual loss is generally not identifiable. We show that under the assumption of strong ignorability, a counterfactual risk is identifiable if and only if the counterfactual loss function is additive in the potential outcomes. Moreover, we demonstrate that additive counterfactual losses can yield treatment recommendations that differ from those based on standard loss functions, provided that the decision problem involves more than two treatment options.

Problem

• Extends decision theory to include counterfactual outcomes for evaluation
• Identifies conditions for counterfactual risk under strong ignorability
• Shows additive losses alter treatment choices in multi-option scenarios

Innovation

• Extends decision theory with counterfactual outcomes
• Identifies risk under strong ignorability assumption
• Uses additive counterfactual loss functions