GenCAD-3D: CAD Program Generation using Multimodal Latent Space Alignment and Synthetic Dataset Balancing

2025-09-17

AI Summary

Automatically generating compilable CAD programs—i.e., parameterized command sequences—from unstructured 3D geometric data (e.g., point clouds or meshes)—remains challenging due to scarce annotated data, insufficient coverage of complex topologies, and heavy reliance on manual intervention. Method: We propose GenCAD-3D, a novel framework comprising three key components: (1) multimodal contrastive learning to align geometric and CAD semantic latent spaces; (2) SynthBal, a synthetic data augmentation strategy that systematically enriches high-complexity CAD program samples; and (3) a latent diffusion model for high-fidelity CAD sequence generation and retrieval. Results: Experiments demonstrate substantial improvements in geometric reconstruction accuracy and significant reduction in invalid output rates. GenCAD-3D consistently outperforms existing baselines across high-complexity scenarios. To foster reproducibility and community advancement, we will open-source our code and dataset.

Technology Category

• Computer Vision: 3D Computer Vision
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Deep Generative Models & Autoencoders

Application Category

• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

CAD programs, structured as parametric sequences of commands that compile into precise 3D geometries, are fundamental to accurate and efficient engineering design processes. Generating these programs from nonparametric data such as point clouds and meshes remains a crucial yet challenging task, typically requiring extensive manual intervention. Current deep generative models aimed at automating CAD generation are significantly limited by imbalanced and insufficiently large datasets, particularly those lacking representation for complex CAD programs. To address this, we introduce GenCAD-3D, a multimodal generative framework utilizing contrastive learning for aligning latent embeddings between CAD and geometric encoders, combined with latent diffusion models for CAD sequence generation and retrieval. Additionally, we present SynthBal, a synthetic data augmentation strategy specifically designed to balance and expand datasets, notably enhancing representation of complex CAD geometries. Our experiments show that SynthBal significantly boosts reconstruction accuracy, reduces the generation of invalid CAD models, and markedly improves performance on high-complexity geometries, surpassing existing benchmarks. These advancements hold substantial implications for streamlining reverse engineering and enhancing automation in engineering design. We will publicly release our datasets and code, including a set of 51 3D-printed and laser-scanned parts on our project site.

Problem

• Generating parametric CAD programs from nonparametric geometric data
• Addressing dataset imbalance and insufficiency in CAD generation models
• Enhancing representation and accuracy for complex CAD geometries

Innovation

• Multimodal generative framework with contrastive learning
• Synthetic data augmentation balancing complex geometries
• Latent diffusion models for CAD sequence generation