Automated Composition of Agents: A Knapsack Approach for Agentic Component Selection

2025-10-18

AI Summary

Automating agent system construction in dynamic, uncertain environments—where component capabilities are incompletely specified and semantic retrieval is unreliable—requires jointly optimizing functional compatibility, cost, and real-time utility during online selection and composition of agents, tools, and foundation models. Method: We propose an online multi-objective optimization framework inspired by the knapsack problem. It replaces conventional semantic retrieval with real-time capability assessment and utility modeling, and introduces the Composer Agent architecture, which integrates dynamic component testing, online learning, and knapsack-based optimization to enable runtime validation and optimal assembly decisions. Results: Experiments across five benchmark datasets demonstrate substantial improvements: single-agent task success rates increase by up to 31.6%; multi-agent success rises significantly from 37% to 87%; and component acquisition costs are markedly reduced.

Technology Category

• Search and Optimization: Learning to Search
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Multiagent Systems: Adversarial Agents

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Designing effective agentic systems requires the seamless composition and integration of agents, tools, and models within dynamic and uncertain environments. Most existing methods rely on static, semantic retrieval approaches for tool or agent discovery. However, effective reuse and composition of existing components remain challenging due to incomplete capability descriptions and the limitations of retrieval methods. Component selection suffers because the decisions are not based on capability, cost, and real-time utility. To address these challenges, we introduce a structured, automated framework for agentic system composition that is inspired by the knapsack problem. Our framework enables a composer agent to systematically identify, select, and assemble an optimal set of agentic components by jointly considering performance, budget constraints, and compatibility. By dynamically testing candidate components and modeling their utility in real-time, our approach streamlines the assembly of agentic systems and facilitates scalable reuse of resources. Empirical evaluation with Claude 3.5 Sonnet across five benchmarking datasets shows that our online-knapsack-based composer consistently lies on the Pareto frontier, achieving higher success rates at significantly lower component costs compared to our baselines. In the single-agent setup, the online knapsack composer shows a success rate improvement of up to 31.6% in comparison to the retrieval baselines. In multi-agent systems, the online knapsack composer increases success rate from 37% to 87% when agents are selected from an agent inventory of 100+ agents. The substantial performance gap confirms the robust adaptability of our method across diverse domains and budget constraints.

Problem

• Automating agentic component selection using knapsack problem principles
• Optimizing performance and cost in dynamic agent composition
• Addressing limitations of static retrieval methods for component reuse

Innovation

• Automated knapsack framework for agentic system composition
• Dynamic component selection based on performance and budget
• Real-time utility modeling for optimal component assembly