CAD-Llama: Leveraging Large Language Models for Computer-Aided Design Parametric 3D Model Generation

2025-05-07

AI Summary

This work addresses the lack of parametric CAD modeling priors and 3D structural understanding in large language models (LLMs). To this end, we propose the first LLM-based framework for parametric 3D modeling. Methodologically, we design a Structured Parametric CAD Code (SPCC) format, construct a hierarchical semantic annotation pipeline, and introduce a CAD-space-knowledge-driven adaptive pretraining and instruction-tuning paradigm, implemented atop the LLaMA architecture for domain specialization. Our contribution is threefold: (1) the first LLM capable of generating editable, geometrically constrained, and hierarchically topologically consistent parametric 3D model sequences; (2) significant performance gains over autoregressive baselines and general-purpose LLMs across multiple metrics—including syntactic validity, geometric fidelity, and structural coherence; and (3) a foundational step toward AI-powered programmable 3D design.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Large Vision Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Recently, Large Language Models (LLMs) have achieved significant success, prompting increased interest in expanding their generative capabilities beyond general text into domain-specific areas. This study investigates the generation of parametric sequences for computer-aided design (CAD) models using LLMs. This endeavor represents an initial step towards creating parametric 3D shapes with LLMs, as CAD model parameters directly correlate with shapes in three-dimensional space. Despite the formidable generative capacities of LLMs, this task remains challenging, as these models neither encounter parametric sequences during their pretraining phase nor possess direct awareness of 3D structures. To address this, we present CAD-Llama, a framework designed to enhance pretrained LLMs for generating parametric 3D CAD models. Specifically, we develop a hierarchical annotation pipeline and a code-like format to translate parametric 3D CAD command sequences into Structured Parametric CAD Code (SPCC), incorporating hierarchical semantic descriptions. Furthermore, we propose an adaptive pretraining approach utilizing SPCC, followed by an instruction tuning process aligned with CAD-specific guidelines. This methodology aims to equip LLMs with the spatial knowledge inherent in parametric sequences. Experimental results demonstrate that our framework significantly outperforms prior autoregressive methods and existing LLM baselines.

Problem

• Generating parametric CAD sequences using LLMs
• Enhancing LLMs for 3D spatial awareness
• Translating CAD commands into structured code

Innovation

• Hierarchical annotation pipeline for CAD sequences
• Structured Parametric CAD Code (SPCC) format
• Adaptive pretraining with CAD-specific instruction tuning