Infer-and-widen, or not?

2024-08-12

AI Summary

This paper addresses systematic bias and excessive width in the “infer-and-widen” paradigm for selective inference. We systematically analyze its performance across three canonical settings: winner’s curse correction, maximum contrast inference, and post-Lasso inference. Under identical randomization-induced selection events, we provide the first rigorous proof that state-of-the-art infer-and-widen confidence intervals are strictly wider than simple alternatives—even under oracle knowledge of the selection event. Moreover, their coverage–length trade-off remains suboptimal compared to competing approaches. By integrating tools from selective inference, conditional confidence intervals, and bias correction, our analysis demonstrates that infer-and-widen is not universally optimal. These findings yield critical theoretical insights and practical warnings, motivating the development of tighter, adaptive post-selection inference methods.

Technology Category

• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty
• Game Theory and Economic Paradigms: Imperfect Information
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

In recent years, there has been substantial interest in the task of selective inference: inference on a parameter that is selected from the data. Many of the existing proposals fall into what we refer to as the emph{infer-and-widen} framework: they produce symmetric confidence intervals whose midpoints do not account for selection and therefore are biased; thus, the intervals must be wide enough to account for this bias. In this paper, we investigate infer-and-widen approaches in three vignettes: the winner's curse, maximal contrasts, and inference after the lasso. In each of these examples, we show that a state-of-the-art infer-and-widen proposal leads to confidence intervals that are much wider than simple alternatives when all methods are tuned to yield emph{identical} randomized selection events. Furthermore, even an ``oracle'' infer-and-widen confidence interval -- the narrowest possible interval that could be theoretically attained via infer-and-widen -- is often wider than these alternatives.

Problem

• Evaluating biased confidence intervals in selective inference
• Comparing infer-and-widen methods in three specific scenarios
• Demonstrating inefficiency of infer-and-widen versus alternatives

Innovation

• Investigates infer-and-widen framework limitations
• Compares symmetric confidence intervals alternatives
• Demonstrates oracle intervals often wider