Continuous Improvement and Parallel Autonomous Exploration: An LLM-Agent Framework for Searching Large Solution Spaces
This work addresses the challenge of efficiently searching for high-quality, diverse solutions in large-scale structured solution spaces by proposing a novel framework based on large language model (LLM) agents. The framework employs a leaderboard-driven retention set to guide continuous improvement of a single agent, while enabling multiple agents to operate in parallel—autonomously performing analysis, implementation, self-evaluation, and iteration—with coordination handled solely by a dedicated orchestrator agent. By innovatively integrating a continuous-improvement reward loop with fully autonomous parallel exploration, the approach transcends the limitations of conventional single-trajectory optimization paradigms. Evaluated on a product-to-catalog matching task, the single-agent configuration achieves best-in-class coverage rates of 47.8–57.4%, which further improves to 62.8–69.4% with five parallel agents, substantially outperforming the 33.3% baseline.