Training Needs Trustworthy Worlds: Verified Synthetic Web Environments for Agent Learning

2026-08-22

AI Summary

本文解决了合成网络环境不真实的问题，通过构建可执行、可审计且基于后端状态的合成网络环境来提高代理学习的可信度。

Abstract

Web agents promise to automate complex digital workflows, but their training remains limited by synthetic environments that look plausible while hiding broken links, inconsistent states, or infeasible tasks. We address the gap between scalable environment generation and trustworthy agent learning by constructing synthetic web environments that are executable, auditable, and grounded in backend state. Our framework represents each generated website as a structured scaffold of pages, navigation links, database records, state-change markers, and task constraints, then verifies and repairs structural, semantic, consistency, and feasibility defects before policy training. During interaction, ordinary UI transitions are executed deterministically, while persistent backend updates are invoked only through validated state-change markers, enabling dense rewards compiled from verified task-progress predicates. Across 500 synthetic environments spanning six domains, our method reduces task-blocking defects and improves feasible-task rate from 48.6% to 94.8%, while producing stronger PPO policies and improving transfer to WebArena, WebShop, and MiniWoB++ without LLM calls at evaluation time. These results show that verified synthetic environments can serve as a scalable and reliable training substrate for compact web agents, shifting synthetic webagent learning from surface-level plausibility toward executable, state-grounded supervision.

Problem

• Web Agents
• Synthetic Environments
• Trustworthy Learning
• Consistency Defects
• Feasible Tasks

Innovation

• synthetic web environments
• executable and auditable
• backend state grounded
• defect verification and repair
• state-change markers