Divide, Optimize, Merge: Fine-Grained LLM Agent Optimization at Scale

2025-05-06

AI Summary

To address context overflow and deteriorating pattern recognition in LLM optimizers under large-scale training trajectories, this paper proposes the Fine-Grained Optimization (FGO) framework, introducing a novel “divide–optimize–merge” three-stage paradigm: trajectories are dynamically partitioned into subsets, each independently optimized, and results are progressively merged; trajectory sampling and reweighting strategies are further incorporated to enhance representativeness. Evaluated on ALFWorld, LogisticsQA, and GAIA, FGO achieves average performance gains of 1.6–8.6% while reducing prompt token consumption by 56.3%. The method maintains a favorable trade-off between high accuracy and low computational overhead across diverse data scales, significantly improving the scalability and robustness of LLM-based agent optimization.

Technology Category

• Search and Optimization: Learning to Search
• Machine Learning: Optimization
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Agentic search

Abstract

LLM-based optimization has shown remarkable potential in enhancing agentic systems. However, the conventional approach of prompting LLM optimizer with the whole training trajectories on training dataset in a single pass becomes untenable as datasets grow, leading to context window overflow and degraded pattern recognition. To address these challenges, we propose Fine-Grained Optimization (FGO), a scalable framework that divides large optimization tasks into manageable subsets, performs targeted optimizations, and systematically combines optimized components through progressive merging. Evaluation across ALFWorld, LogisticsQA, and GAIA benchmarks demonstrate that FGO outperforms existing approaches by 1.6-8.6% while reducing average prompt token consumption by 56.3%. Our framework provides a practical solution for scaling up LLM-based optimization of increasingly sophisticated agent systems. Further analysis demonstrates that FGO achieves the most consistent performance gain in all training dataset sizes, showcasing its scalability and efficiency.

Problem

• Addresses context window overflow in large-scale LLM optimization
• Improves pattern recognition in fragmented optimization tasks
• Enhances scalability of LLM-based agent system optimization

Innovation

• Divides large tasks into manageable subsets
• Performs targeted optimizations on subsets
• Systematically combines optimized components progressively