Bayesian optimization for mixed variables using an adaptive dimension reduction process: applications to aircraft design

2025-04-11

AI Summary

In multidisciplinary design optimization—particularly aircraft design—the presence of high-dimensional mixed variables (continuous, integer, and categorical) causes combinatorial explosion in the hyperparameter space of Bayesian optimization surrogate models. To address this, we propose a Partial Least Squares (PLS)-based adaptive dimensionality reduction framework. It dynamically learns variable coupling structures to significantly compress the surrogate hyperparameter space, while integrating mixed-variable encoding with an adaptive hyperparameter selection mechanism to balance modeling accuracy and tuning efficiency. Evaluated on analytical benchmarks and two real-world aircraft design cases, our method achieves >30% faster convergence and improves optimal solution quality by 12–18% compared to genetic algorithms, while reducing hyperparameter count by ~60%. Our core contribution is the first application of PLS for hyperparameter pruning in Bayesian optimization, enabling efficient and robust optimization under high-dimensional mixed-variable settings.

Technology Category

• Search and Optimization: Mixed Discrete/Continuous Search
• Machine Learning: Optimization
• Constraint Satisfaction and Optimization: Mixed Discrete/Continuous Optimization

Application Category

• User Modeling, Personalization and Recommendation: ML for personalized search and recommendations
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

Multidisciplinary design optimization methods aim at adapting numerical optimization techniques to the design of engineering systems involving multiple disciplines. In this context, a large number of mixed continuous, integer and categorical variables might arise during the optimization process and practical applications involve a large number of design variables. Recently, there has been a growing interest in mixed variables constrained Bayesian optimization but most existing approaches severely increase the number of the hyperparameters related to the surrogate model. In this paper, we address this issue by constructing surrogate models using less hyperparameters. The reduction process is based on the partial least squares method. An adaptive procedure for choosing the number of hyperparameters is proposed. The performance of the proposed approach is confirmed on analytical tests as well as two real applications related to aircraft design. A significant improvement is obtained compared to genetic algorithms.

Problem

• Optimizing mixed continuous, integer, categorical variables in engineering design
• Reducing hyperparameters in surrogate models for Bayesian optimization
• Improving aircraft design optimization compared to genetic algorithms

Innovation

• Bayesian optimization with adaptive dimension reduction
• Surrogate models using fewer hyperparameters
• Partial least squares for hyperparameter selection