Conformal Prediction with Upper and Lower Bound Models

2025-03-06

AI Summary

This work addresses the poor coverage of conventional conformal prediction (CP) in regression tasks where the target variable is only known to lie within deterministic upper and lower bounds—e.g., estimating optimal values of optimization problems—particularly in regions where these bounds are tight. To overcome the limitation of global, fixed-threshold post-hoc calibration, we propose CPUL, a model selection framework that jointly optimizes model architecture and calibration strategy for region-aware prediction interval construction, and OMLT, an adaptive thresholding mechanism that dynamically adjusts thresholds over nested intervals to mitigate coverage degradation near tight boundaries. Together, they establish a dual correction paradigm: “model selection + region-adaptive thresholding.” Experiments on large-scale optimization value bounding tasks demonstrate that our approach significantly improves the trade-off between coverage guarantee and interval width, consistently outperforming state-of-the-art CP baselines.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Stochastic Optimization
• Constraint Satisfaction and Optimization: Constraint Programming

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

This paper studies a Conformal Prediction (CP) methodology for building prediction intervals in a regression setting, given only deterministic lower and upper bounds on the target variable. It proposes a new CP mechanism (CPUL) that goes beyond post-processing by adopting a model selection approach over multiple nested interval construction methods. Paradoxically, many well-established CP methods, including CPUL, may fail to provide adequate coverage in regions where the bounds are tight. To remedy this limitation, the paper proposes an optimal thresholding mechanism, OMLT, that adjusts CPUL intervals in tight regions with undercoverage. The combined CPUL-OMLT is validated on large-scale learning tasks where the goal is to bound the optimal value of a parametric optimization problem. The experimental results demonstrate substantial improvements over baseline methods across various datasets.

Problem

• Develops Conformal Prediction for regression with bounded targets
• Introduces CPUL and OMLT to improve interval coverage
• Validates method on large-scale parametric optimization tasks

Innovation

• Introduces CPUL for nested interval construction
• Proposes OMLT for optimal thresholding in tight regions
• Validates CPUL-OMLT on large-scale learning tasks