Hephaestus: Improving Fundamental Agent Capabilities of Large Language Models through Continual Pre-Training

2025-02-10

AI Summary

Large language model (LLM) agents suffer from fundamental limitations in API invocation, internal reasoning, and adaptation to environmental feedback. Method: This paper introduces the first agent-centric continual pretraining paradigm, built upon Hephaestus-Forge—a large-scale, agent-oriented corpus comprising 103B tokens and covering 76,537 APIs—synthesizing tool documentation and real-world API invocation trajectories. It further identifies the scaling law governing optimal data mixture ratios to refine training protocols. The approach integrates continual pretraining, API trajectory modeling, and explicit tool knowledge injection. Contribution/Results: Evaluated on three major agent benchmarks, the resulting model substantially outperforms open-source LLMs of small-to-medium scale and matches commercial LLMs in agent capabilities, achieving significant improvements in API call accuracy, reasoning coherence, and environmental adaptability.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Natural Language Processing: (Large) Language Models

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Due to the scarcity of agent-oriented pre-training data, LLM-based autonomous agents typically rely on complex prompting or extensive fine-tuning, which often fails to introduce new capabilities while preserving strong generalizability. We introduce Hephaestus-Forge, the first large-scale pre-training corpus designed to enhance the fundamental capabilities of LLM agents in API function calling, intrinsic reasoning and planning, and adapting to environmental feedback. Hephaestus-Forge comprises 103B agent-specific data encompassing 76,537 APIs, including both tool documentation to introduce knowledge of API functions and function calling trajectories to strengthen intrinsic reasoning. To explore effective training protocols, we investigate scaling laws to identify the optimal recipe in data mixing ratios. By continual pre-training on Hephaestus-Forge, Hephaestus outperforms small- to medium-scale open-source LLMs and rivals commercial LLMs on three agent benchmarks, demonstrating the effectiveness of our pre-training corpus in enhancing fundamental agentic capabilities and generalization of LLMs to new tasks or environments.

Problem

• Enhance LLM agent capabilities
• Optimize pre-training data mixing
• Improve generalization to new tasks

Innovation

• Continual pre-training corpus
• API function calling enhancement
• Scaling laws optimization