On the Brittleness of Maximum Likelihood Estimation for Gaussian Process Hyperparameter Optimization

2026-08-13

AI Summary

This study addresses the fragility of maximum likelihood estimation (MLE) in Gaussian process hyperparameter optimization by proposing theoretical evaluation metrics and robust alternative training strategies. Systematically elucidating the failure mechanisms of MLE, this work establishes a novel hyperparameter optimization framework. Experimental results demonstrate that the proposed method significantly outperforms tabular foundation models in predictive accuracy, uncertainty quantification, and inference efficiency. Furthermore, it effectively enhances the robustness of downstream tasks such as Bayesian optimization. Collectively, these contributions provide a solid theoretical and methodological foundation for the reliable application of Gaussian processes, mitigating critical vulnerabilities associated with traditional MLE approaches in practical scenarios.

Abstract

Machine learning (ML) has become an indispensable part of modern engineering design workflows. A crucial step in training an ML model is the selection of the loss function which can be systematically formulated via various techniques such as maximum likelihood estimation (MLE) and cross-validation . While MLE is one of the most popular, effective, and intuitive mechanisms for training ML models, it is brittle: if the assumptions underpinning it are not met, the trained ML model may generalize poorly. This brittleness affects even Gaussian processes (GPs) which are widely used in engineering design and are often (incorrectly) presumed to be very robust to overfitting. In this paper, we fundamentally evaluate the brittleness of MLE in the context of training GPs for probabilistic regression or classification tasks. We compare theoretically grounded metrics against MLE and propose practical solutions. Our extensive studies demonstrate the effectiveness of our solutions in downstream design tasks such as Bayesian optimization and provide a blueprint for practitioners to build accurate and robust GPs that can even outperform tabular foundation models in terms of prediction accuracy, uncertainty quantification, and inference cost. Our contributions are publicly available via GitHub at https://github.com/Bostanabad-Research-Group/GP-vs-TabPFN-vs-GPyTorch.

Problem

• Maximum Likelihood Estimation
• Gaussian Process
• Brittleness
• Hyperparameter Optimization
• Overfitting

Innovation

• Gaussian Process
• Maximum Likelihood Estimation
• Hyperparameter Optimization
• Brittleness
• Uncertainty Quantification