Text-to-CadQuery: A New Paradigm for CAD Generation with Scalable Large Model Capabilities

2025-05-10

AI Summary

This paper addresses the challenge of zero-shot natural language-to-parametric CAD modeling. It reformulates the task as direct text-to-CadQuery domain-specific code generation, bypassing error-prone intermediate representations. The key contributions are: (1) construction of the first large-scale CadQuery instruction dataset (170K samples), empirically confirming a positive correlation between LLM scale and CAD generation performance; (2) supervised fine-tuning of open-source foundation models—including Qwen, Llama, and Phi—using CadQuery’s Python API as the unified output representation; and (3) a joint optimization strategy integrating spatial semantic parsing with code generation. Experiments demonstrate that the best-performing model achieves a 69.3% Top-1 exact match accuracy—a 10.5 percentage-point improvement—and reduces Chamfer distance by 48.6%, significantly outperforming existing text-to-CAD approaches.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Diffusion Models for Vision

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Computer-aided design (CAD) is fundamental to modern engineering and manufacturing, but creating CAD models still requires expert knowledge and specialized software. Recent advances in large language models (LLMs) open up the possibility of generative CAD, where natural language is directly translated into parametric 3D models. However, most existing methods generate task-specific command sequences that pretrained models cannot directly handle. These sequences must be converted into CAD representations such as CAD vectors before a 3D model can be produced, which requires training models from scratch and adds unnecessary complexity. To tackle this issue, we propose generating CadQuery code directly from text, leveraging the strengths of pretrained LLMs to produce 3D models without intermediate representations, using this Python-based scripting language. Since LLMs already excel at Python generation and spatial reasoning, fine-tuning them on Text-to-CadQuery data proves highly effective. Given that these capabilities typically improve with scale, we hypothesize that larger models will perform better after fine-tuning. To enable this, we augment the Text2CAD dataset with 170,000 CadQuery annotations. We fine-tune six open-source LLMs of varying sizes and observe consistent improvements. Our best model achieves a top-1 exact match of 69.3%, up from 58.8%, and reduces Chamfer Distance by 48.6%. Project page: https://github.com/Text-to-CadQuery/Text-to-CadQuery.

Problem

• Generating CAD models from text without expert knowledge
• Eliminating intermediate representations in CAD generation
• Improving accuracy of text-to-CAD conversion with LLMs

Innovation

• Generates CadQuery code directly from text
• Leverages pretrained LLMs for 3D modeling
• Augments dataset with 170,000 CadQuery annotations