Transfer Learning Architectures for Scalable Multi-Fidelity Bayesian Optimization

2026-07-25

AI Summary

This work addresses the limitations of conventional Gaussian processes in multi-fidelity Bayesian optimization for molecules and materials, particularly their poor computational scalability and difficulty handling non-smooth search spaces. The authors propose a transfer learning–based surrogate model that leverages abundant low-cost data to learn generalizable representations, which are then transferred to sparse high-cost data to drive closed-loop optimization. They systematically demonstrate, for the first time, the efficacy of transfer learning as a core surrogate in multi-fidelity Bayesian optimization and introduce a mean-greedy acquisition strategy as an alternative to traditional uncertainty-based exploration. Across experiments ranging from synthetic benchmarks to real-world molecular and materials tasks, the proposed method consistently outperforms 11 transfer learning architectures and 4 Gaussian process baselines, achieving superior solutions with lower computational overhead and establishing itself as a preferred surrogate model for such problems.

Abstract

Self-driving laboratories increasingly rely on multi-fidelity Bayesian optimization (MFBO) to balance cheap, approximate evaluations against scarce, expensive ones, with a predictive surrogate at its core. Gaussian processes (GPs) are the default choice, but they scale poorly as data accumulate and assume a smooth landscape that molecular and materials search spaces routinely violate. Transfer learning offers an alternative suited to this regime: it learns a representation from abundant cheap data and adapts it to sparse expensive data. Despite its use in property prediction, transfer learning has not been tested as the engine of a closed-loop optimization. Here we benchmark eleven transfer-learning surrogates against four GP methods under an identical selection rule, fidelity budget, and model size, across nine tasks spanning synthetic functions to real chemistry and materials problems. GPs win on smooth, low-dimensional functions but perform worst on molecular and materials problems, where transfer-learning surrogates reach substantially better solutions using far less computation. Because acquisition policy is held fixed across surrogates, this advantage is attributable to the surrogate itself. Uncertainty-driven exploration is not reliably beneficial, and calibration does not predict optimization performance, so greedy exploitation of the transfer-learned mean is the more robust default. Transfer learning is therefore the surrogate of choice for molecular and materials MFBO.

Problem

• multi-fidelity Bayesian optimization
• transfer learning
• surrogate models
• molecular optimization
• materials discovery

Innovation

• transfer learning
• multi-fidelity Bayesian optimization
• surrogate modeling
• self-driving laboratories
• molecular design