Winners with Confidence: Discrete Argmin Inference with an Application to Model Selection

2024-08-04

AI Summary

This paper addresses the problem of reliably identifying the index with the minimal mean from noisy observations—a task central to model selection, policy comparison, and discrete maximum likelihood estimation. To handle high-dimensional settings, frequent ties, and globally dependent data, we propose a test statistic grounded in asymptotic normality. Methodologically, we innovatively integrate cross-validation with differential privacy mechanisms to establish a central limit theorem applicable to non-independent data, and design an adaptive hyperparameter tuning strategy that balances bias and variance. Theoretically, our approach guarantees statistical consistency. Empirically, it significantly improves selection stability and confidence in both synthetic and real-world experiments. Overall, this work provides a new framework for discrete parameter inference under noise—one that unifies theoretical rigor with practical robustness.

Technology Category

• Reasoning under Uncertainty: Stochastic Optimization
• Machine Learning: Calibration & Uncertainty Quantification
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Security and Privacy: Large-scale security measurements

Abstract

We study the problem of finding the index of the minimum value of a vector from noisy observations. This problem is relevant in population/policy comparison, discrete maximum likelihood, and model selection. We develop an asymptotically normal test statistic, even in high-dimensional settings and with potentially many ties in the population mean vector, by integrating concepts and tools from cross-validation and differential privacy. The key technical ingredient is a central limit theorem for globally dependent data. We also propose practical ways to select the tuning parameter that adapts to the signal landscape. Numerical experiments and data examples demonstrate the ability of the proposed method to achieve a favorable bias-variance trade-off in practical scenarios.

Problem

• Finding index of minimum value from noisy observations
• Developing asymptotically normal test statistic for high-dimensional data
• Proposing practical tuning parameter selection for signal adaptation

Innovation

• Asymptotically normal test statistic development
• Integration of cross-validation and differential privacy
• Central limit theorem for globally dependent data