Can Bayesian Optimization Efficiently Find a Strong Single Expert in Neural Thickets?

2026-08-11

AI Summary

This work addresses the critical challenge of efficiently selecting high-performing single experts under limited evaluation budgets in post-training scenarios, where existing gradient-free methods are often hindered by high evaluation costs. We propose a novel approach that integrates Bayesian optimization with random linear embedding, leveraging a Gaussian process surrogate model to guide the search within a low-dimensional weight subspace—entirely without backpropagation. To our knowledge, this is the first application of Bayesian optimization with low-dimensional embeddings in gradient-free post-training, substantially reducing the number of required evaluations. Experimental results on the Qwen2.5-Instruct model series (0.5B–3B) demonstrate that our method achieves comparable or superior reasoning performance using only one-fifth the evaluation budget of RandOpt.

Abstract

Gradient-free post-training has emerged as a compelling alternative to gradient-based optimization for large language models (LLMs), but existing approaches remain costly. We ask whether structured search can identify a strong single expert under a modest evaluation budget. Motivated by evidence that useful weight updates lie in low-dimensional subspaces, we apply Bayesian optimization within a random linear embedding of weight space. Our method requires no backpropagation and uses a Gaussian process surrogate to guide candidate evaluations efficiently. Across several reasoning benchmarks with Qwen2.5-Instruct models from 0.5B to 3B parameters, Bayesian optimization using five times less candidate evaluations matches or exceeds RandOpt. These results show that surrogate-guided search can substantially reduce the evaluation cost of gradient-free post-training while producing stronger deployable single experts.

Problem

• Bayesian Optimization
• Gradient-free Post-training
• Single Expert
• Evaluation Efficiency
• Neural Thickets

Innovation

• Bayesian optimization
• gradient-free post-training
• low-dimensional subspace
• Gaussian process
• structured search