Layered Image Vectorization via Semantic Simplification

2024-06-08

arXiv.org

AI Summary

This paper addresses three key challenges in raster-to-vector conversion: weak semantic alignment, hierarchical redundancy, and low visual fidelity. To this end, we propose a progressive hierarchical vectorization framework that first constructs a semantically aligned macro-structure and then progressively refines details across layers, yielding compact, layered vector representations. Our core contributions are: (1) a semantic simplification mechanism leveraging the feature-averaging effect of Score Distillation Sampling for progressive structural simplification; (2) a two-stage “structure-first, detail-later” vectorization paradigm; and (3) an inter-layer optimization strategy enforcing dual alignment—explicit structural alignment and implicit texture/style alignment. Extensive experiments demonstrate that our method significantly outperforms state-of-the-art approaches across diverse image categories, producing SVGs with fewer layers, stronger semantic consistency, higher visual fidelity, and superior editability.

Technology Category

• Computer Vision: Learning & Optimization for CV
• Machine Learning: Structured Learning
• Search and Optimization: Distributed Search

Application Category

• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

This work presents a progressive image vectorization technique that reconstructs the raster image as layer-wise vectors from semantic-aligned macro structures to finer details. Our approach introduces a new image simplification method leveraging the feature-average effect in the Score Distillation Sampling mechanism, achieving effective visual abstraction from the detailed to coarse. Guided by the sequence of progressive simplified images, we propose a two-stage vectorization process of structural buildup and visual refinement, constructing the vectors in an organized and manageable manner. The resulting vectors are layered and well-aligned with the target image's explicit and implicit semantic structures. Our method demonstrates high performance across a wide range of images. Comparative analysis with existing vectorization methods highlights our technique's superiority in creating vectors with high visual fidelity, and more importantly, achieving higher semantic alignment and more compact layered representation. The project homepage is https://szuviz.github.io/layered_vectorization/.

Problem

• Progressive image vectorization from semantic-aligned structures to details.
• Two-stage vectorization: structural buildup and visual refinement.
• Achieves high semantic alignment and compact layered representation.

Innovation

• Progressive image vectorization via semantic simplification
• Two-stage vectorization: structural buildup and refinement
• Layered vectors aligned with semantic structures