Department of Industrial Engineering & Decision Analytics [Joint IEDA/ISOM seminar] - Optimizing the Input Data Collection for Ranking and Selection

We investigate a Bayesian ranking and selection (R&S) problem under input uncertainty when all solutions share a collection of independent input models. These Bayesian input models are updated as more data is collected from multiple independent sources ultimately converging to the data-generating distributions. Our goal is to design a sequential sampling algorithm that identifies the true optimum under the data-generating distributions most efficiently given a data collection budget. Two types of data collections are considered: input data acquisition from each source and simulation sampling for a solution at a particular set of input models. We propose the most probable best (MPB) as a Bayesian estimator of the optimum and characterize its probabilistic convergence rate to the optimum focusing on the case when there is a finite set of candidates for the true input distributions. We derive the optimal asymptotic static sampling ratios for the input data collection and simulation by maximizing the rate function. Based on this analysis, a sequential sampling algorithm is proposed, which is further bolstered by combining the kernel ridge regression to improve the mean prediction at the solution-input model pairs. We benchmark our algorithm against a state-of-the-art method that considers the same R&S problem.