Towards Internet-Scale Training For Agents

2025-02-10

AI Summary

Training web navigation agents at internet scale heavily relies on costly, labor-intensive human annotation. Method: This paper introduces the first fully LLM-driven, zero-human-annotation training paradigm. Leveraging Llama 3.1 70B, it establishes a multi-stage collaborative architecture that autonomously generates tasks, executes navigation trajectories, and evaluates success rates—forming a closed-loop pipeline integrated with feasibility verification and harmful-content filtering. The framework scales to over one million web tasks across 150,000 real-world websites. Results: It achieves a task-solving rate of 16.7%; hybrid training boosts step accuracy by up to 122.1%; and cross-site generalization on WebLINX improves by 149.0%. This work pioneers large-scale, trustworthy, and scalable training of web agents without human intervention.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Learning to Search
• Multiagent Systems: Adversarial Agents

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

The predominant approach for training web navigation agents gathers human demonstrations for a set of popular websites and hand-written tasks, but it is becoming clear that human data are an inefficient resource. We develop a pipeline to facilitate Internet-scale training for agents without laborious human annotations. In the first stage, an LLM generates tasks for 150k diverse websites. In the next stage, LLM agents complete tasks and produce trajectories. In the final stage, an LLM reviews the trajectories and judges their success. Language models are competitive with human annotators, detecting and filtering out harmful content with an accuracy of 97%, generating feasible tasks with an 89% rate, and judging successful trajectories with an 82.6% accuracy. Scaling the pipeline, agents based on Llama 3.1 70B solve 16.7% of tasks for 150k sites. Training on the data generated by our pipeline is competitive with training on human demonstrations. In data-limited settings derived from Mind2Web and WebLINX, we improve Step Accuracy by up to +89.5% and +122.1% respectively for agents trained on mixtures of data from our pipeline, and human data. When training agents with all available human data from these benchmarks, agents fail to generalize to diverse real sites, and adding our data improves their generalization by +149.0% for WebLINX and +156.3% for Mind2Web. Code will be available at: data-for-agents.github.io.

Problem

• Internet-scale training for web navigation agents
• Reducing reliance on human annotations
• Improving agent generalization to diverse websites

Innovation

• LLM generates diverse website tasks
• LLM agents produce task trajectories
• LLM reviews and judges trajectory success