Relation Reasoning with LLMs in Expensive Optimization

2026-04-30

AI Summary

This work addresses the challenges of expensive optimization problems—characterized by high evaluation costs, absence of gradients, and limited budgets—by proposing R2SAEA, a novel approach that introduces large language models (LLMs) into surrogate-assisted evolutionary optimization for relational reasoning. R2SAEA achieves zero-shot surrogate modeling without per-generation retraining through iterative anchor-based context construction, linearized prompting, and a voting-based relation prediction aggregation mechanism. Built upon Qwen2.5 and enhanced with GRPO-based reinforcement fine-tuning and quantization for deployment, the method demonstrates superior relational prediction accuracy and optimization performance on both single- and multi-objective benchmarks compared to existing surrogate-assisted evolutionary algorithms and general-purpose LLMs, while enabling efficient edge deployment.

Abstract

Expensive optimization problems (EOPs) are black-box tasks with costly objective evaluations and no gradient access, making the evaluation budget the key bottleneck. Surrogate-assisted evolutionary algorithms (SAEAs) reduce evaluations via surrogate predictions, but conventional surrogates often require frequent retraining as populations evolve, incurring overhead. This paper proposes R2SAEA, a reinforcement-trained relation-based large language model (LLM) surrogate assisted evolutionary algorithm. We cast relation-based surrogate modeling as an in-context pairwise reasoning task. To enable efficient inference in evolutionary loops, we develop an anchor-based iterative context construction strategy that reduces prompt complexity from quadratic to linear in population size, and a voting-based aggregation scheme that converts predicted relations into scores for offspring selection. We further build an RL pipeline from evolutionary trajectories and fine-tune Qwen2.5 with GRPO. Experiments on single- and multi-objective benchmarks show improved relation prediction and state-of-the-art optimization performance over strong SAEA baselines and general LLMs. Quantization also enables efficient edge deployment, supporting a zero-shot surrogate paradigm without per-generation retraining. Code and models are available at https://github.com/Septend9/R2SAEA.

Problem

• Expensive Optimization
• Surrogate-assisted Evolutionary Algorithms
• Black-box Optimization
• Evaluation Budget
• Relation Reasoning

Innovation

• relation reasoning
• large language model
• surrogate-assisted evolutionary algorithm
• reinforcement learning
• expensive optimization