Automated CAD Modeling Sequence Generation from Text Descriptions via Transformer-Based Large Language Models

2025-05-26

AI Summary

To address the labor-intensive, low-efficiency, and error-prone nature of manual CAD modeling in industrial detailed design, this paper proposes an end-to-end natural language–driven CAD generation framework. Methodologically: (1) a semi-automated data annotation pipeline is developed to alleviate the scarcity of labeled CAD operation sequences; (2) TCADGen—a dual-channel feature-aggregation Transformer—is designed to jointly model parametric constraints and visual semantics; (3) a confidence-guided fine-tuning mechanism integrates large language models (LLMs) and vision-language LMs (VLLMs) for robust sequence generation. Experiments demonstrate that our approach significantly outperforms conventional methods in both generation accuracy for complex models and inference efficiency. It enables concurrent parametric and appearance-aware modeling, supporting practical industrial design workflows. The source code is publicly available.

Technology Category

• Natural Language Processing: Generation
• Machine Learning: Deep Generative Models & Autoencoders
• Computer Vision: Diffusion Models for Vision

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Designing complex computer-aided design (CAD) models is often time-consuming due to challenges such as computational inefficiency and the difficulty of generating precise models. We propose a novel language-guided framework for industrial design automation to address these issues, integrating large language models (LLMs) with computer-automated design (CAutoD).Through this framework, CAD models are automatically generated from parameters and appearance descriptions, supporting the automation of design tasks during the detailed CAD design phase. Our approach introduces three key innovations: (1) a semi-automated data annotation pipeline that leverages LLMs and vision-language large models (VLLMs) to generate high-quality parameters and appearance descriptions; (2) a Transformer-based CAD generator (TCADGen) that predicts modeling sequences via dual-channel feature aggregation; (3) an enhanced CAD modeling generation model, called CADLLM, that is designed to refine the generated sequences by incorporating the confidence scores from TCADGen. Experimental results demonstrate that the proposed approach outperforms traditional methods in both accuracy and efficiency, providing a powerful tool for automating industrial workflows and generating complex CAD models from textual prompts. The code is available at https://jianxliao.github.io/cadllm-page/

Problem

• Automating CAD model generation from text descriptions
• Improving accuracy and efficiency in industrial design workflows
• Integrating LLMs with CAD for precise modeling sequences

Innovation

• LLM and VLLM for semi-automated data annotation
• Transformer-based CAD generator with dual-channel features
• CADLLM refines sequences using confidence scores