Online Prediction with Limited Selectivity

2025-08-13

AI Summary

This work investigates optimal error bounds for prediction with limited selectivity (PLS), where predictors may issue forecasts only on a specified subset of time-series instances. To formalize this constrained online prediction setting, we propose a PLS framework and introduce a novel instance-dependent complexity measure. Based on this measure, we derive tight upper and lower bounds on the optimal prediction error. Our theoretical analysis integrates instance-specific complexity characterization with average-case analysis, yielding the first exact error characterization within the PLS paradigm. Furthermore, we prove that these bounds are achievable with high probability on randomly generated instances, thereby validating both the effectiveness and tightness of the proposed complexity metric. Collectively, our results establish a new theoretical foundation for selective prediction under resource constraints, advancing the understanding of fundamental limits in selective forecasting.

Technology Category

• Machine Learning: Online Learning & Bandits
• Constraint Satisfaction and Optimization: Other Foundations of Constraint Satisfaction
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Social Networks and Social Media: Influence propagation, information diffusion, and the prediction on networks
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

Selective prediction [Dru13, QV19] models the scenario where a forecaster freely decides on the prediction window that their forecast spans. Many data statistics can be predicted to a non-trivial error rate without any distributional assumptions or expert advice, yet these results rely on that the forecaster may predict at any time. We introduce a model of Prediction with Limited Selectivity (PLS) where the forecaster can start the prediction only on a subset of the time horizon. We study the optimal prediction error both on an instance-by-instance basis and via an average-case analysis. We introduce a complexity measure that gives instance-dependent bounds on the optimal error. For a randomly-generated PLS instance, these bounds match with high probability.

Problem

• Modeling prediction with restricted start times
• Analyzing optimal error in limited selectivity
• Introducing instance-dependent complexity measures

Innovation

• Limited Selectivity Prediction model introduced
• Instance-dependent error bounds analyzed
• Random PLS instances match bounds