How much Data do We Need? Sequential Data Collection for Stochastic Programming

2026-07-11

AI Summary

This study addresses a central challenge in data-driven optimization: determining the optimal stopping time for data acquisition under parameter uncertainty by balancing sampling costs against information gains. The authors propose a Bayesian learning–based sequential data collection framework that explicitly models the trade-off between information gain and sampling cost, enabling a reward-driven adaptive stopping mechanism that jointly optimizes data acquisition and decision-making. Integrating Bayesian parameter updating, sequential decision theory, and stochastic programming, the approach formulates multiple stopping strategies within the newsvendor model. Numerical experiments demonstrate that the proposed strategies significantly reduce redundant sampling compared to fixed-budget and ex post optimal benchmarks while maintaining near-optimal decision performance.

Abstract

Data-driven optimization often requires collecting data to estimate uncertain model parameters before solving the underlying decision problem. In practice, however, data acquisition may incur non-negligible costs, making it critical to determine when to stop additional data collection. In this paper, we study an optimal stopping problem for sequential data collection in stochastic optimization under parameter uncertainty. We propose a benefit-driven stopping framework that balances information gain and sampling cost. We model the unknown distribution parameter within a Bayesian learning framework and update beliefs sequentially as new observations are collected. At each iteration, the decision maker evaluates the expected marginal benefit of additional data relative to the unit sampling cost and determines whether to continue sampling or stop and implement the optimization decision. Based on this framework, we develop several stopping policies. The proposed policies are evaluated through a newsvendor problem with exponentially distributed demand. Numerical experiments compare the policies with fixed-budget and hindsight benchmark strategies. The results show that benefit-driven stopping rules can substantially reduce unnecessary data collection while achieving near-optimal decision performance, demonstrating the effectiveness of adaptive stopping in data-driven optimization.

Problem

• sequential data collection
• stochastic programming
• optimal stopping
• parameter uncertainty
• data-driven optimization

Innovation

• sequential data collection
• optimal stopping
• Bayesian learning
• stochastic programming
• benefit-driven stopping