Structural Evaluation Metrics for SVG Generation via Leave-One-Out Analysis

2026-04-09

AI Summary

Existing evaluation methods for SVG generation overlook structural editability and struggle to assess element-wise contributions and modularity. This work proposes the first structured evaluation framework based on Leave-One-Out analysis, enabling fine-grained assessment by iteratively removing individual elements and quantifying their impact on the overall visual output. The framework introduces zero-shot artifact detection and a concept-to-element attribution mechanism, and defines four structural metrics: purity, coverage, compactness, and locality. Validated across five generative systems, three complexity levels, and over 19,000 editing trials, the proposed metrics demonstrate both effectiveness and practical utility, establishing an interpretable and editable paradigm for evaluating SVG generation quality.

Technology Category

• Machine Learning: Evaluation and Analysis
• Search and Optimization: Evaluation and Analysis
• Computer Vision: Segmentation

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Web data quality in the era of algorithmically-generated content
• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success

Abstract

Scalable Vector Graphics (SVG) represent visual content as structured, editable code. Each element (path, shape, or text node) can be individually inspected, transformed, or removed. This structural editability is a main motivation for SVG generation, yet prevailing evaluation protocols primarily reduce the output to a single similarity score against a reference image or input texts, measuring how faithfully the result reproduces an image or follows the instructions, but not how well it preserves the structural properties that make SVG valuable. In particular, existing metrics cannot determine which generated elements contribute positively to overall visual quality, how visual concepts map to specific parts of the code, or whether the generated output supports meaningful downstream editing. We introduce element-level leave-one-out (LOO) analysis, inspired by the classic jackknife estimator. The procedure renders the SVG with and without each element, measures the resulting visual change, and derives a suite of structural quality metrics. Despite its simplicity, the jackknife's capacity to decompose an aggregate statistic into per-sample contributions translates directly to this setting. From a single mechanism, we obtain: (1) quality scores per element through LOO scoring that enable zero-shot artifact detection; (2) concept-element attribution that maps each element to the visual concept it serves; and (3) four structural metrics, purity, coverage, compactness, and locality, that quantify SVG modularity from complementary perspectives. We validate these metrics on over 19,000 edits (5 types) across 5 generation systems and 3 complexity tiers.

Problem

• SVG generation
• structural evaluation
• element-level analysis
• visual quality
• editability

Innovation

• leave-one-out analysis
• structural evaluation
• SVG generation
• element-level attribution
• modularity metrics