An Axiomatic Approach to Comparing Sensitivity Parameters

2025-04-29

AI Summary

This paper addresses the lack of objective criteria for selecting sensitivity analysis methods in omitted-variable bias assessment. Existing approaches rely on untestable assumptions, rendering empirical comparison impossible; conventional “interpretability” standards are inherently subjective. To resolve this, we propose the first axiomatic framework for omitted-variable sensitivity analysis. Its core innovation is the introduction of the *parameter consistency* axiom—defined with respect to the sampling distribution of covariates—which yields a formal, distribution-based criterion for method comparability. We rigorously falsify widely used methods (e.g., Rosenbaum’s bounding approach) and identify several novel procedures satisfying consistency. By integrating causal inference, counterfactual modeling, and sampling distribution theory, our framework shifts method selection from subjective judgment to formal, testable validation—establishing the first empirically verifiable principle for robust causal inference under unmeasured confounding. (149 words)

Technology Category

• Reasoning under Uncertainty: Causality
• Constraint Satisfaction and Optimization: Other Foundations of Constraint Satisfaction
• Machine Learning: Causal Learning

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Many methods are available for assessing the importance of omitted variables. These methods typically make different, non-falsifiable assumptions. Hence the data alone cannot tell us which method is most appropriate. Since it is unreasonable to expect results to be robust against all possible robustness checks, researchers often use methods deemed"interpretable", a subjective criterion with no formal definition. In contrast, we develop the first formal, axiomatic framework for comparing and selecting among these methods. Our framework is analogous to the standard approach for comparing estimators based on their sampling distributions. We propose that sensitivity parameters be selected based on their covariate sampling distributions, a design distribution of parameter values induced by an assumption on how covariates are assigned to be observed or unobserved. Using this idea, we define a new concept of parameter consistency, and argue that a reasonable sensitivity parameter should be consistent. We prove that the literature's most popular approach is inconsistent, while several alternatives are consistent.

Problem

• Comparing sensitivity parameters for omitted variables
• Formal axiomatic framework for method selection
• Ensuring parameter consistency in sensitivity analysis

Innovation

• Axiomatic framework for comparing sensitivity methods
• Selection based on covariate sampling distributions
• Introduction of parameter consistency concept