Doc2Agent: Scalable Generation of Tool-Using Agents from API Documentation

2025-06-24

AI Summary

Real-world API documentation is often unstructured and highly heterogeneous across tools, leading to high development costs and poor generalization in agent construction. Method: This paper proposes an end-to-end, scalable tool generation framework: (1) parsing raw API documentation to automatically extract interface semantics and parameter constraints; (2) generating executable Python tool functions; and (3) incorporating a code-agent-driven iterative feedback optimization mechanism to improve tool invocation accuracy. Contribution/Results: To our knowledge, this is the first approach achieving high validation rates (>92%) in fully automated tool generation directly from real-world API documentation—without human annotation or domain-specific adaptation. On the WebArena benchmark, it improves task success rate by 55% and reduces tool construction cost by 90%. Furthermore, it demonstrates strong cross-domain generalization, validated in complex vertical domains such as sugar materials science.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Constraint Satisfaction and Optimization: Solvers and Tools
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

REST APIs play important roles in enriching the action space of web agents, yet most API-based agents rely on curated and uniform toolsets that do not reflect the complexity of real-world APIs. Building tool-using agents for arbitrary domains remains a major challenge, as it requires reading unstructured API documentation, testing APIs and inferring correct parameters. We propose Doc2Agent, a scalable pipeline to build agents that can call Python-based tools generated from API documentation. Doc2Agent generates executable tools from API documentations and iteratively refines them using a code agent. We evaluate our approach on real-world APIs, WebArena APIs, and research APIs, producing validated tools. We achieved a 55% relative performance improvement with 90% lower cost compared to direct API calling on WebArena benchmark. A domain-specific agent built for glycomaterial science further demonstrates the pipeline's adaptability to complex, knowledge-rich tasks. Doc2Agent offers a generalizable solution for building tool agents from unstructured API documentation at scale.

Problem

• Generating scalable agents from unstructured API documentation
• Automating tool creation and refinement for diverse APIs
• Improving performance and cost efficiency in API-based agents

Innovation

• Generates tools from API documentation automatically
• Iteratively refines tools using a code agent
• Achieves high performance with lower cost