Multi-Task Surrogate-Assisted Search with Bayesian Competitive Knowledge Transfer for Expensive Optimization

2025-10-27

AI Summary

For expensive optimization problems (EOPs), where function evaluations are severely limited and surrogate-assisted search (SAS) suffers from cold-start and negative transfer issues, this paper proposes a multi-task surrogate-assisted search framework. Its core innovation is a Bayesian competitive knowledge transfer mechanism that, for the first time, unifies prior beliefs and empirical evidence within a Bayesian framework to dynamically assess knowledge transferability. By incorporating intra- and inter-task solution competition, the mechanism adaptively suppresses negative transfer while enhancing positive transfer. The framework is algorithm-agnostic—compatible with diverse SAS methods—and enables efficient knowledge reuse without additional function evaluations. Empirical results demonstrate that the approach significantly reduces evaluation overhead and accelerates convergence across multi-task EOP benchmarks, with validation in real-world engineering applications.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Reasoning under Uncertainty: Stochastic Optimization
• Constraint Satisfaction and Optimization: Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

Expensive optimization problems (EOPs) present significant challenges for traditional evolutionary optimization due to their limited evaluation calls. Although surrogate-assisted search (SAS) has become a popular paradigm for addressing EOPs, it still suffers from the cold-start issue. In response to this challenge, knowledge transfer has been gaining popularity for its ability to leverage search experience from potentially related instances, ultimately facilitating head-start optimization for more efficient decision-making. However, the curse of negative transfer persists when applying knowledge transfer to EOPs, primarily due to the inherent limitations of existing methods in assessing knowledge transferability. On the one hand, a priori transferability assessment criteria are intrinsically inaccurate due to their imprecise understandings. On the other hand, a posteriori methods often necessitate sufficient observations to make correct inferences, rendering them inefficient when applied to EOPs. Considering the above, this paper introduces a Bayesian competitive knowledge transfer (BCKT) method developed to improve multi-task SAS (MSAS) when addressing multiple EOPs simultaneously. Specifically, the transferability of knowledge is estimated from a Bayesian perspective that accommodates both prior beliefs and empirical evidence, enabling accurate competition between inner-task and inter-task solutions, ultimately leading to the adaptive use of promising solutions while effectively suppressing inferior ones. The effectiveness of our method in boosting various SAS algorithms for both multi-task and many-task problems is empirically validated, complemented by comparative studies that demonstrate its superiority over peer algorithms and its applicability to real-world scenarios. The source code of our method is available at https://github.com/XmingHsueh/MSAS-BCKT.

Problem

• Addresses expensive optimization problems with limited evaluation calls
• Solves cold-start issues in surrogate-assisted search methods
• Mitigates negative transfer in knowledge transfer for optimization

Innovation

• Bayesian competitive knowledge transfer for multi-task optimization
• Adaptive solution competition between inner and inter tasks
• Empirical validation on multi-task and many-task problems