UniVector: Unified Vector Extraction via Instance-Geometry Interaction

2025-10-15

AI Summary

Existing vector extraction methods typically target isolated structural primitives (e.g., polygons or polylines) and model instance semantics and geometric attributes independently, limiting their capacity to represent complex, mixed-structure scenes. This paper introduces the first unified vector extraction framework, enabling end-to-end joint modeling and generation of heterogeneous vector primitives—including polygons, polylines, and line segments—via structured query encoding, instance-geometry interactive context updating, and dynamic shape constraints. Crucially, it is the first approach to jointly optimize semantic categories, topological relationships, and geometric shapes within a single model, substantially enhancing expressiveness for intricate structures. The framework achieves state-of-the-art performance on both single- and multi-structure vector extraction tasks. To foster further research, we also release Multi-Vector, the first diverse, large-scale benchmark dataset for vector graphics understanding and generation.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Structured Learning
• Natural Language Processing: Information Extraction

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Vector extraction retrieves structured vector geometry from raster images, offering high-fidelity representation and broad applicability. Existing methods, however, are usually tailored to a single vector type (e.g., polygons, polylines, line segments), requiring separate models for different structures. This stems from treating instance attributes (category, structure) and geometric attributes (point coordinates, connections) independently, limiting the ability to capture complex structures. Inspired by the human brain's simultaneous use of semantic and spatial interactions in visual perception, we propose UniVector, a unified VE framework that leverages instance-geometry interaction to extract multiple vector types within a single model. UniVector encodes vectors as structured queries containing both instance- and geometry-level information, and iteratively updates them through an interaction module for cross-level context exchange. A dynamic shape constraint further refines global structures and key points. To benchmark multi-structure scenarios, we introduce the Multi-Vector dataset with diverse polygons, polylines, and line segments. Experiments show UniVector sets a new state of the art on both single- and multi-structure VE tasks. Code and dataset will be released at https://github.com/yyyyll0ss/UniVector.

Problem

• Unifies vector extraction for multiple geometry types
• Overcomes independent treatment of semantic and geometric attributes
• Enables single-model handling of polygons, polylines and segments

Innovation

• Unified vector extraction via instance-geometry interaction
• Structured queries encode instance and geometry information
• Dynamic shape constraint refines global structures and key points