🤖 AI Summary
Existing large language models generate HTML pages that render well statically but often fail under user interactions such as scrolling, clicking, or window resizing—issues largely undetectable by conventional screenshot-based evaluation. This work proposes HTMLCure, a novel framework that leverages browser interaction trajectories as the basis for both assessment and repair. By executing pages across multiple viewports and interactive states, recording deterministic behavioral evidence, and employing a vision-language model to analyze key-frame trajectories, HTMLCure drives a state-aware, closed-loop repair engine. The approach substantially enhances the interactive robustness and usability of generated web pages. Built upon 97K raw prompts, the authors curate a high-quality supervised fine-tuning dataset of 40K samples. The refined HTMLCure-27B-Refined model achieves a score of 50.6 on HTMLBench-400 (with a 45.2% deterministic test pass rate) and 81.2 on MiniAppBench, outperforming baselines by 15.3 points on average.
📝 Abstract
LLMs can now produce full HTML pages, but many of those pages are only superficially correct: they render once, then fail under scroll, hover, click, resize, or gameplay. Evaluation from screenshots can miss these failures, and filtering discards many pages that are still repairable. We introduce HTMLCure, a browser experience framework that evaluates HTML after the system has interacted with it. The evaluator executes the page across viewports and interaction states, records deterministic browser evidence, and gives the VLM curated keyframes from the executed trajectory rather than isolated screenshots. The same state signal drives a closed loop repair engine: HTMLCure diagnoses the current page, chooses a state specific repair family, runs each candidate again, and exports quality cleared pages for SFT. On a 97K prompt corpus, this expands the directly usable seed into a candidate pool of 63703 quality cleared pages, from which we construct the final refined SFT set of 40K pages. Under the same backbone and training recipe, HTMLCure-27B-Refined reaches 50.6 on HTMLBench-400 with 45.2% deterministic test case pass, placing it in the same performance band as strong reference rows such as Kimi-K2.6 and GPT-5.4. On the released MiniAppBench validation split, it reaches 81.2 average, improving raw 27B SFT by 15.3 points and approaching the level of strong reference systems.