SVG-Score: Human-Aligned Evaluation of Text-to-SVG Generation

2026-09-03

AI Summary

本文针对SVG生成缺乏领域特定评估协议的问题，提出了一种与人类判断对齐的评估框架，通过改进CLIP评分和训练视觉-语言模型来提高评估准确性。

Abstract

Scalable Vector Graphics (SVG) generation is attracting increasing attention as generative models improve in expressiveness and controllability. Progress, however, is held back by the lack of domain-specific evaluation protocols: current practice relies on metrics designed for natural images, most notably CLIPScore, which was never trained on vector graphics and aligns only partially with human judgment. We introduce \textbf{\ours}, a human-aligned evaluation framework for text-to-SVG generation. Through controlled caption and image perturbations, we first show that CLIP-based scores barely react to the errors SVG generators actually make, such as wrong colors, counts, and spatial relations, and that off-the-shelf Vision-Language Model (VLM) judges, while more sensitive, respond unevenly across error types and SVG styles. We then introduce a human-annotated dataset for \textit{Semantic Alignment}, measuring how faithfully a generated SVG reflects its caption. Building on it, we develop two complementary evaluators: CLIP scorers adapted to vector graphics and then aligned to human preferences, for fast large-scale evaluation, and a VLM judge trained with supervised fine-tuning and reward-shaped reinforcement learning, for more expressive and interpretable assessment. Using both, we benchmark major open-source, commercial, and optimization-based SVG generators on an independent caption set.

Problem

• SVG generation
• evaluation protocol
• CLIPScore
• human judgment
• vector graphics

Innovation

• human-aligned evaluation
• SVG generation
• CLIPScore adaptation
• Semantic Alignment
• reinforcement learning