LongWebBench: Evaluating Structural and Functional Webpage Generation in Long-Horizon Settings

2026-06-16

AI Summary

Current vision-language models struggle to simultaneously maintain structural coherence and interactive functionality when generating long webpages, particularly under multi-screen, dynamic, and long-range scenarios. This work proposes LongWebBench—the first benchmark for joint evaluation of structure and function in long webpage generation—comprising 490 real-world long webpages and 507 multi-step interaction tasks. By integrating VLM-driven multi-dimensional structural similarity metrics, DOM tree parsing, end-to-end agent-based interaction validation, and human consistency analysis, the authors establish an executability-centered automatic evaluation protocol. Experiments reveal that state-of-the-art VLMs suffer from declining structural fidelity as webpage length increases, often producing visually plausible but functionally non-executable pages. This study introduces a new evaluation paradigm and a publicly available dataset for long webpage generation.

Abstract

Recent vision-language models (VLMs) have shown promising progress in generating webpages from visual inputs, yet existing evaluations mainly focus on short, single-screen, and largely static webpages. We introduce LongWebBench, a benchmark for evaluating long-horizon webpage generation from both structural and functional perspectives. LongWebBench contains 490 real-world long webpages for structural fidelity evaluation and 507 goal-oriented interaction tasks over 129 webpages for functional evaluation. It employs two complementary protocols: a multi-dimensional VLM-based metric for assessing long-range structural coherence, and a DOM-augmented agent-based pipeline for end-to-end functional verification. We further examine the automatic evaluation protocols through human agreement analysis. Experiments with state-of-the-art open-source and proprietary VLMs under single-image and multi-image settings reveal that structural fidelity degrades as webpage length increases, while visually plausible generations often fail to support executable multi-step interactions. These results highlight the need to evaluate long webpage generation beyond visual similarity, with executable interaction as a core criterion. Our code and data are available at https://github.com/zheny2751-dotcom/LongWebBench.

Problem

• long-horizon webpage generation
• structural fidelity
• functional evaluation
• vision-language models
• executable interaction

Innovation

• LongWebBench
• webpage generation
• structural fidelity
• functional evaluation
• vision-language models