CAD-Recode: Reverse Engineering CAD Code from Point Clouds

2024-12-18

arXiv.org

AI Summary

This work addresses the end-to-end inverse engineering problem of reconstructing parametric CAD models from point clouds. Methodologically, it introduces the first approach that directly generates executable Python code encoding CAD modeling procedures—comprising sketches and operation sequences—via a lightweight point cloud projector jointly trained with a fine-tuned small-scale LLM in a collaborative decoding architecture; training is enabled by a million-scale synthetic CAD code dataset. Key contributions include: (1) the first explicit encoding of CAD semantic modeling as structured, executable Python programs; (2) high-fidelity geometric reconstruction, achieving a Chamfer distance tenfold lower than DeepCAD and Fusion360 baselines; (3) substantially reduced point cloud density requirements; and (4) generated code compatible with commercial LLMs, enabling downstream CAD editing and question-answering tasks.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Deep Generative Models & Autoencoders
• Computer Vision: 3D Computer Vision

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Computer-Aided Design (CAD) models are typically constructed by sequentially drawing parametric sketches and applying CAD operations to obtain a 3D model. The problem of 3D CAD reverse engineering consists of reconstructing the sketch and CAD operation sequences from 3D representations such as point clouds. In this paper, we address this challenge through novel contributions across three levels: CAD sequence representation, network design, and dataset. In particular, we represent CAD sketch-extrude sequences as Python code. The proposed CAD-Recode translates a point cloud into Python code that, when executed, reconstructs the CAD model. Taking advantage of the exposure of pre-trained Large Language Models (LLMs) to Python code, we leverage a relatively small LLM as a decoder for CAD-Recode and combine it with a lightweight point cloud projector. CAD-Recode is trained solely on a proposed synthetic dataset of one million diverse CAD sequences. CAD-Recode significantly outperforms existing methods across three datasets while requiring fewer input points. Notably, it achieves 10 times lower mean Chamfer distance than state-of-the-art methods on DeepCAD and Fusion360 datasets. Furthermore, we show that our CAD Python code output is interpretable by off-the-shelf LLMs, enabling CAD editing and CAD-specific question answering from point clouds.

Problem

• Reconstruct CAD sketch-extrude sequences from point clouds.
• Translate point clouds into executable Python CAD code.
• Enable CAD editing and question answering via LLMs.

Innovation

• Translates point clouds into Python CAD code
• Uses pre-trained LLMs for CAD sequence decoding
• Trained on a million CAD sequences dataset