Beyond Browsing: API-Based Web Agents

2024-10-21

arXiv.org

AI Summary

This work addresses the poor robustness of AI agents in complex web tasks caused by overreliance on browser-based interaction. To this end, we propose a novel API-first interaction paradigm. Methodologically, we design a task-agnostic API-first hybrid agent that integrates LLM-driven API discovery and planning, dynamic schema parsing, and unified modeling of multi-source action spaces; we further construct a systematic evaluation framework using WebArena. Our contributions are threefold: (1) the first systematic empirical validation that APIs can serve as viable mainstream alternatives to browser-based web interaction; (2) the first general-purpose, task-agnostic hybrid agent supporting both API and browser modalities without task-specific fine-tuning; and (3) achieving a 35.8% task success rate on WebArena—surpassing pure browser-based agents by over 20.0 percentage points and establishing a new state-of-the-art for task-agnostic web agents.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Multiagent Systems: Adversarial Agents
• Cognitive Modeling & Cognitive Systems: Agent Architectures

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Agentic search
• Economics, Online Markets and Human Computation: Sustainability of Web economics

Abstract

Web browsers are a portal to the internet, where much of human activity is undertaken. Thus, there has been significant research work in AI agents that interact with the internet through web browsing. However, there is also another interface designed specifically for machine interaction with online content: application programming interfaces (APIs). In this paper we ask -- what if we were to take tasks traditionally tackled by browsing agents, and give AI agents access to APIs? To do so, we propose two varieties of agents: (1) an API-calling agent that attempts to perform online tasks through APIs only, similar to traditional coding agents, and (2) a Hybrid Agent that can interact with online data through both web browsing and APIs. In experiments on WebArena, a widely-used and realistic benchmark for web navigation tasks, we find that API-based agents outperform web browsing agents. Hybrid Agents out-perform both others nearly uniformly across tasks, resulting in a more than 20.0% absolute improvement over web browsing alone, achieving a success rate of 35.8%, achiving the SOTA performance among task-agnostic agents. These results strongly suggest that when APIs are available, they present an attractive alternative to relying on web browsing alone.

Problem

• AI Performance
• API vs Browser
• Complex Network Operations

Innovation

• AI Models
• API Integration
• Enhanced Navigation