Rubrics as Visual-Repair Context for Self-Evolving UI-to-Code Generation

2026-08-25

AI Summary

本文针对UI-to-代码生成中视觉修复不稳定的问题，提出了一种基于评分标准引导的自我进化框架RubSE，通过结构化的视觉修复上下文来指导每次修订，以实现更稳定的改进过程。

Abstract

Large vision-language models have shown strong progress in UI-to-code generation, yet their test-time self-evolution remains unstable. We first identify a fundamental obstacle, termed visual repair coupling: a local code edit may propagate through layout, style, and component dependencies, correcting one visual mismatch while degrading regions that were previously faithful. To address this issue, we present RubSE, a Rubric-guided Self-Evolution framework that uses rubrics to represent visual feedback as a structured visual-repair context. At each refinement round, RubSE generates typed candidate rubrics, selects one prioritized repair target, and stores previously selected rubrics as history, thereby steering each revision toward a well-scoped visual repair while discouraging repeated or over-broad changes. Evaluations across six VLMs and three UI-to-code benchmarks demonstrate that RubSE substantially outperforms naïve self-evolution in final-round and best-round settings, achieving more stable refinement trajectories and a higher trajectory-level performance ceiling. Further analysis shows that RubSE mitigates trajectory collapse by improving recovery from severe visual regressions, and that stronger rubric generators can transfer effective visual-repair guidance to weaker code improvers.

Problem

• visual repair coupling
• self-evolution
• UI-to-code generation

Innovation

• UI-to-code generation
• Self-Evolution
• Visual Repair
• Rubric-guided
• Stable Refinement