Generating Human-AI Collaborative Design Sequence for 3D Assets via Differentiable Operation Graph

2025-08-25

AI Summary

A representational gap exists between 3D-AIGC and human-centered design paradigms: AI systems predominantly rely on neural rendering (e.g., NeRF, Gaussian splatting) or mesh representations, whereas designers use parametric CAD tools, hindering efficient human-AI collaboration. Method: We propose the first differentiable operation graph framework for editable 3D asset generation, modeling fundamental modeling operations (e.g., extrusion, Boolean) as differentiable units. Our hierarchical graph neural network incorporates gating mechanisms to jointly optimize continuous and discrete parameters. Training is fully unsupervised—without ground-truth operation sequences—using Chamfer distance loss, sequence-length regularization, and domain-specific rule penalties. Contribution/Results: The generated operation sequences achieve high geometric fidelity, topological validity, step-level interpretability, and strong editability. Critically, outputs are natively compatible with industrial CAD software, enabling seamless integration into professional design workflows and significantly enhancing human-AI co-design efficiency.

Technology Category

• Humans and AI: Game Design — Procedural Content Generation & Storytelling
• Computer Vision: Diffusion Models for Vision
• Cognitive Modeling & Cognitive Systems: Computational Creativity

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

The emergence of 3D artificial intelligence-generated content (3D-AIGC) has enabled rapid synthesis of intricate geometries. However, a fundamental disconnect persists between AI-generated content and human-centric design paradigms, rooted in representational incompatibilities: conventional AI frameworks predominantly manipulate meshes or neural representations (emph{e.g.}, NeRF, Gaussian Splatting), while designers operate within parametric modeling tools. This disconnection diminishes the practical value of AI for 3D industry, undermining the efficiency of human-AI collaboration. To resolve this disparity, we focus on generating design operation sequences, which are structured modeling histories that comprehensively capture the step-by-step construction process of 3D assets and align with designers' typical workflows in modern 3D software. We first reformulate fundamental modeling operations (emph{e.g.}, emph{Extrude}, emph{Boolean}) into differentiable units, enabling joint optimization of continuous (emph{e.g.}, emph{Extrude} height) and discrete (emph{e.g.}, emph{Boolean} type) parameters via gradient-based learning. Based on these differentiable operations, a hierarchical graph with gating mechanism is constructed and optimized end-to-end by minimizing Chamfer Distance to target geometries. Multi-stage sequence length constraint and domain rule penalties enable unsupervised learning of compact design sequences without ground-truth sequence supervision. Extensive validation demonstrates that the generated operation sequences achieve high geometric fidelity, smooth mesh wiring, rational step composition and flexible editing capacity, with full compatibility within design industry.

Problem

• Bridging AI-generated 3D content with human design workflows
• Converting mesh/neural representations to parametric modeling sequences
• Enabling differentiable optimization of design operation parameters

Innovation

• Differentiable modeling operations for gradient-based optimization
• Hierarchical graph with gating mechanism for end-to-end learning
• Unsupervised sequence learning with domain constraints and rules