Empowering Vector Graphics with Consistently Arbitrary Viewing and View-dependent Visibility

2025-05-27

AI Summary

This work addresses three key challenges in text-to-vector graphics generation: cross-view inconsistency, coarse geometric detail, and difficulty in modeling occlusion relationships. We propose a dual-branch collaborative optimization framework: a primary branch generates differentiable vector graphics, while an auxiliary branch incorporates 3D Gaussian splatting to provide geometric and visibility priors. We further integrate classifier-free guidance scheduling with a visibility-aware rendering module to enable view-consistent, progressive detail refinement and dynamic stroke culling. To our knowledge, this is the first approach to achieve viewpoint-dependent occlusion awareness and dynamic stroke clipping in vector graphic synthesis. Experimental results demonstrate significant improvements in cross-view consistency for 3D sketches and icons, multi-granularity detail fidelity, and occlusion modeling accuracy.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Planning, Routing, and Scheduling: Planning with Language Models
• Machine Learning: Deep Generative Models & Autoencoders

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

This work presents a novel text-to-vector graphics generation approach, Dream3DVG, allowing for arbitrary viewpoint viewing, progressive detail optimization, and view-dependent occlusion awareness. Our approach is a dual-branch optimization framework, consisting of an auxiliary 3D Gaussian Splatting optimization branch and a 3D vector graphics optimization branch. The introduced 3DGS branch can bridge the domain gaps between text prompts and vector graphics with more consistent guidance. Moreover, 3DGS allows for progressive detail control by scheduling classifier-free guidance, facilitating guiding vector graphics with coarse shapes at the initial stages and finer details at later stages. We also improve the view-dependent occlusions by devising a visibility-awareness rendering module. Extensive results on 3D sketches and 3D iconographies, demonstrate the superiority of the method on different abstraction levels of details, cross-view consistency, and occlusion-aware stroke culling.

Problem

• Generates 3D vector graphics from text prompts
• Ensures cross-view consistency and detail control
• Improves occlusion awareness in vector graphics

Innovation

• Dual-branch optimization framework for vector graphics
• 3D Gaussian Splatting for domain gap bridging
• Visibility-awareness rendering for occlusion handling