Localized conformal model selection

2026-02-22

AI Summary

This work addresses the challenge of achieving adaptive model selection that simultaneously ensures finite-sample coverage validity and accommodates spatial heterogeneity without distributional assumptions. The authors propose a localized conformal model selection framework that, during the calibration phase, symmetrically constructs upper and lower surrogate intervals to form a data-dependent safety index set for identifying favorable models. This approach enables local adaptivity while preserving sample exchangeability. Notably, it is the first method to unify local adaptivity with post-selection validity, guaranteeing exact marginal coverage while flexibly adapting to data heterogeneity. Empirical results demonstrate that under strong heterogeneity or low-noise conditions, the resulting prediction intervals are significantly shorter than those produced by a fixed oracle model.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Local Search

Application Category

• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

We propose a localized conformal model selection framework that integrates local adaptivity with post-selection validity for distribution-free prediction. By performing model selection symmetrically across calibration points using upper and lower surrogate intervals, we construct a data-dependent safe index set that contains the oracle model and preserves exchangeability. The resulting ensemble procedure retains exact finite-sample marginal coverage while adapting to spatial heterogeneity and model complexity. Simulations demonstrate substantial reductions in interval length compared to the best fixed model, especially in heterogeneous and low-noise settings.

Problem

• conformal prediction
• model selection
• distribution-free inference
• spatial heterogeneity
• post-selection validity

Innovation

• localized conformal inference
• model selection
• distribution-free prediction
• spatial heterogeneity
• finite-sample coverage