Vector Scaffolding: Inter-Scale Orchestration for Differentiable Image Vectorization

2026-05-12

AI Summary

Existing differentiable vector graphics methods suffer from an imbalance between area and boundary gradients, leading to topological collapse during optimization and producing redundant, non-editable “polygon soups.” To address this, this work proposes a hierarchical optimization framework that jointly refines multi-scale curves through interior-point gradient aggregation, a progressive layering strategy, and an accelerated warm-up learning rate schedule. By transforming flat pixel-wise matching into structured topological construction, the method substantially enhances optimization stability and efficiency. Experiments on image vectorization demonstrate a 2.5× speedup over the current state-of-the-art while achieving up to a 1.4 dB improvement in PSNR.

Abstract

Differentiable vector graphics have enabled powerful gradient-based optimization of vector primitives directly from raster images. However, existing frameworks formulate this as a flat optimization problem, forcing hundreds to thousands of randomly initialized curves to blindly compete for pixel-level error reduction. This disordered optimization leads to topology collapse, where macroscopic structures are distorted by internal high-frequency noise, resulting in a redundant and uneditable "polygon soup" that limits practical editability. To address this limitation, we propose Vector Scaffolding, a novel hierarchical optimization framework that shifts from flat pixel-matching to structured topological construction tailored for vector graphics. By identifying a key cause of topology collapse as the mathematical imbalance between area and boundary gradients, we introduce Interior Gradient Aggregation to stabilize the learning dynamics of multi-scale curve mixtures. Upon this stabilized landscape, we employ Progressive Stratification and Rapid Inflation Scheduling to progressively densify vector primitives with extremely high learning rates ($\times 50$). Experiments demonstrate that our approach accelerates optimization by $2.5\times$ while simultaneously improving PSNR by up to 1.4 dB over the previous state of the art.

Problem

• differentiable vectorization
• topology collapse
• vector graphics
• optimization disorder
• editability

Innovation

• Vector Scaffolding
• Differentiable Vectorization
• Interior Gradient Aggregation
• Progressive Stratification
• Topology Collapse