INST-Sculpt: Interactive Stroke-based Neural SDF Sculpting

2025-02-05

AI Summary

Neural implicit signed distance functions (SDFs) struggle to support intuitive, continuous surface sculpting. Method: We propose the first stroke-driven interactive sculpting framework for neural implicit surfaces. Our approach introduces a tubular neighborhood sampling strategy and a parameterizable brush profile to map user-drawn strokes into smooth, localized 3D deformation fields. Within the implicit space, we directly optimize the SDF via a gradient-guided weight update mechanism, preserving sign consistency and surface smoothness without mesh conversion. Contribution/Results: Unlike existing methods limited to discrete point editing, ours enables millisecond-latency, continuous geometric operations—including indentation and protrusion—while maintaining topological flexibility. Experiments demonstrate significant improvements over baselines in complex topology deformation and fine-detail enhancement, achieving both high fidelity and real-time interactivity.

Technology Category

• Search and Optimization: Mixed Discrete/Continuous Search
• Machine Learning: Learning with Manifolds
• Computer Vision: 3D Computer Vision

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

Recent advances in implicit neural representations have made them a popular choice for modeling 3D geometry, achieving impressive results in tasks such as shape representation, reconstruction, and learning priors. However, directly editing these representations poses challenges due to the complex relationship between model weights and surface regions they influence. Among such editing tools, sculpting, which allows users to interactively carve or extrude the surface, is a valuable editing operation to the graphics and modeling community. While traditional mesh-based tools like ZBrush facilitate fast and intuitive edits, a comparable toolkit for sculpting neural SDFs is currently lacking. We introduce a framework that enables interactive surface sculpting edits directly on neural implicit representations. Unlike previous works limited to spot edits, our approach allows users to perform stroke-based modifications on the fly, ensuring intuitive shape manipulation without switching representations. By employing tubular neighborhoods to sample strokes and custom brush profiles, we achieve smooth deformations along user-defined curves, providing precise control over the sculpting process. Our method demonstrates that intricate and versatile edits can be made while preserving the smooth nature of implicit representations.

Problem

• Interactive editing of neural SDFs
• Stroke-based surface sculpting
• Preserving smooth implicit representations

Innovation

• Interactive neural SDF sculpting
• Stroke-based surface modifications
• Tubular neighborhoods sampling strokes