WIR3D: Visually-Informed and Geometry-Aware 3D Shape Abstraction

2025-05-07

AI Summary

This work addresses the problem of abstracting 3D shapes into sparse, visually meaningful curves while preserving geometric structure and multi-view visual features (e.g., texture) and enabling user-controllable editing. To this end, we propose WIR3D—a two-stage Bézier-curve-based optimization framework. Our method innovatively leverages intermediate CLIP activations to guide curve generation, introduces a spatially localized keypoint loss for user-driven feature control, and enforces neural signed distance function (SDF) constraints to ensure surface fidelity and the effectiveness of curves as deformation handles. The framework jointly optimizes curve parameters, distills CLIP features, and incorporates local keypoint supervision. Extensive evaluation on diverse complex 3D models demonstrates high-fidelity abstraction, significantly advancing shape simplification, artistic modeling, and interactive editing tasks.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Learning with Manifolds
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Responsible Web: Data and user privacy-enhancing technologies for the Web
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

We present WIR3D, a technique for abstracting 3D shapes through a sparse set of visually meaningful curves in 3D. We optimize the parameters of Bezier curves such that they faithfully represent both the geometry and salient visual features (e.g. texture) of the shape from arbitrary viewpoints. We leverage the intermediate activations of a pre-trained foundation model (CLIP) to guide our optimization process. We divide our optimization into two phases: one for capturing the coarse geometry of the shape, and the other for representing fine-grained features. Our second phase supervision is spatially guided by a novel localized keypoint loss. This spatial guidance enables user control over abstracted features. We ensure fidelity to the original surface through a neural SDF loss, which allows the curves to be used as intuitive deformation handles. We successfully apply our method for shape abstraction over a broad dataset of shapes with varying complexity, geometric structure, and texture, and demonstrate downstream applications for feature control and shape deformation.

Problem

• Abstracting 3D shapes using visually meaningful curves
• Optimizing Bezier curves for geometry and visual features
• Enabling user control over abstracted features and deformation

Innovation

• Optimizes Bezier curves for 3D shape abstraction
• Uses CLIP activations to guide optimization process
• Employs neural SDF loss for surface fidelity