WebGen-Agent: Enhancing Interactive Website Generation with Multi-Level Feedback and Step-Level Reinforcement Learning

2025-09-26

AI Summary

Existing LLM-based web code generation methods rely solely on code execution feedback, failing to assess visual fidelity and interactive quality. This work proposes Step-GRPO, a hierarchical visual feedback and step-wise reinforcement learning framework: it integrates multimodal feedback from vision-language models (VLMs), combines GUI agent scoring, screenshot-based visual analysis, and backtracking-based action selection to enable fine-grained process supervision and closed-loop optimization. Crucially, it introduces explicit visual understanding into the generation pipeline, overcoming the limitations of syntax- or execution-only evaluation. On WebGen-Bench, Step-GRPO boosts Claude-3.5-Sonnet’s task accuracy from 26.4% to 51.9% (with an appearance score of 3.9) and improves Qwen2.5-Coder-7B’s accuracy to 45.4% (appearance score 3.7) after fine-tuning—both substantially surpassing prior state-of-the-art methods.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

Agent systems powered by large language models (LLMs) have demonstrated impressive performance on repository-level code-generation tasks. However, for tasks such as website codebase generation, which depend heavily on visual effects and user-interaction feedback, current code agents rely only on simple code execution for feedback and verification. This approach fails to capture the actual quality of the generated code. In this paper, we propose WebGen-Agent, a novel website-generation agent that leverages comprehensive and multi-level visual feedback to iteratively generate and refine the website codebase. Detailed and expressive text descriptions and suggestions regarding the screenshots and GUI-agent testing of the websites are generated by a visual language model (VLM), together with scores that quantify their quality. The screenshot and GUI-agent scores are further integrated with a backtracking and select-best mechanism, enhancing the performance of the agent. Utilizing the accurate visual scores inherent in the WebGen-Agent workflow, we further introduce extit{Step-GRPO with Screenshot and GUI-agent Feedback} to improve the ability of LLMs to act as the reasoning engine of WebGen-Agent. By using the screenshot and GUI-agent scores at each step as the reward in Step-GRPO, we provide a dense and reliable process supervision signal, which effectively improves the model's website-generation ability. On the WebGen-Bench dataset, WebGen-Agent increases the accuracy of Claude-3.5-Sonnet from 26.4% to 51.9% and its appearance score from 3.0 to 3.9, outperforming the previous state-of-the-art agent system. Additionally, our Step-GRPO training approach increases the accuracy of Qwen2.5-Coder-7B-Instruct from 38.9% to 45.4% and raises the appearance score from 3.4 to 3.7.

Problem

• Enhancing website generation with multi-level visual feedback
• Improving code quality through step-level reinforcement learning
• Addressing limitations of current code execution verification methods

Innovation

• Uses multi-level visual feedback for website refinement
• Integrates screenshot and GUI testing with backtracking mechanism
• Applies step-level reinforcement learning with visual rewards