OPT-BENCH: Evaluating the Iterative Self-Optimization of LLM Agents in Large-Scale Search Spaces

2026-05-09

AI Summary

This study investigates whether large language models (LLMs) can achieve continuous self-improvement in dynamic environments through intrinsic cognitive mechanisms, rather than relying solely on memory or external tools. To this end, we introduce OPT-BENCH, a benchmark comprising 20 machine learning tasks and 10 NP-hard problems, along with OPT-Agent—a framework that emulates the human cognitive loop by integrating perception, memory, and reasoning into a closed feedback cycle. Evaluating 19 prominent LLMs, our experiments reveal that more capable models better leverage environmental feedback for iterative refinement, yet their performance remains fundamentally constrained by the underlying model capacity and falls short of human expert levels. This work presents the first systematic assessment of LLMs’ intrinsic adaptability in complex problem-solving, offering a novel paradigm for autonomous agent evolution.

Abstract

Large Language Models (LLMs) have demonstrated remarkable capabilities in reasoning and tool use. However, the fundamental cognitive faculties essential for problem solving, including perception, reasoning, and memory, remain the stable core of intelligence. Unlike memorizing specific patterns, humans succeed in novel environments by applying these intrinsic faculties to adapt and optimize. Yet, whether LLMs possess this essential capacity, namely the ability to continuously refine solutions in response to dynamic environmental feedback, remains underexplored. To address this challenge, we introduce OPT-BENCH, a benchmark for evaluating self-improvement capabilities in large-scale search spaces. By combining 20 machine learning tasks with 10 classic NP-hard problems, OPT-BENCH provides a rigorous setting to assess whether agents can adapt through intrinsic self-reflection rather than rote tool application. We further propose OPT-Agent, a framework that emulates human-like cognitive adaptation. It operates through a general perception, memory, and reasoning loop, iteratively refining solutions based on environmental feedback. Through extensive experiments on 19 LLMs from 7 model families, including reasoning models, general models, and open-source models ranging from 3B to 235B parameters, we demonstrate that stronger models are more effective at leveraging feedback signals for self-improvement. However, this upper-bound adaptability remains fundamentally constrained by the models' base capacity, and even the most advanced LLMs still fall short of human expert performance.

Problem

• self-optimization
• large language models
• adaptive reasoning
• search space
• environmental feedback

Innovation

• self-optimization
• large language models
• cognitive adaptation
• iterative refinement
• benchmarking