Pointer-CAD: Unifying B-Rep and Command Sequences via Pointer-based Edges & Faces Selection

2026-03-04

AI Summary

This work proposes Pointer-CAD, a novel framework that introduces a pointer mechanism into large language model (LLM)-driven CAD generation to address critical limitations in existing command-sequence-based approaches. Current methods struggle to support explicit selection of geometric entities—such as faces and edges—and suffer from topological errors due to discretization of continuous variables, hindering complex editing operations. Pointer-CAD overcomes these challenges by decomposing the modeling process into stepwise operations, where each step jointly leverages textual instructions and the preceding B-Rep model to explicitly select geometric entities via pointers, thereby guiding precise command generation. This approach unifies B-Rep geometry representation with command sequences, enabling robust entity interaction and substantially reducing quantization-induced topological and segmentation errors. Evaluated on a dataset of 575,000 samples, Pointer-CAD demonstrates superior capability in efficiently generating complex CAD structures compared to state-of-the-art methods.

Technology Category

• Natural Language Processing: Generation
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Diffusion Models for Vision

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Constructing computer-aided design (CAD) models is labor-intensive but essential for engineering and manufacturing. Recent advances in Large Language Models (LLMs) have inspired the LLM-based CAD generation by representing CAD as command sequences. But these methods struggle in practical scenarios because command sequence representation does not support entity selection (e.g. faces or edges), limiting its ability to support complex editing operations such as chamfer or fillet. Further, the discretization of a continuous variable during sketch and extrude operations may result in topological errors. To address these limitations, we present Pointer-CAD, a novel LLM-based CAD generation framework that leverages a pointer-based command sequence representation to explicitly incorporate the geometric information of B-rep models into sequential modeling. In particular, Pointer-CAD decomposes CAD model generation into steps, conditioning the generation of each subsequent step on both the textual description and the B-rep generated from previous steps. Whenever an operation requires the selection of a specific geometric entity, the LLM predicts a Pointer that selects the most feature-consistent candidate from the available set. Such a selection operation also reduces the quantization error in the command sequence-based representation. To support the training of Pointer-CAD, we develop a data annotation pipeline that produces expert-level natural language descriptions and apply it to build a dataset of approximately 575K CAD models. Extensive experimental results demonstrate that Pointer-CAD effectively supports the generation of complex geometric structures and reduces segmentation error to an extremely low level, achieving a significant improvement over prior command sequence methods, thereby significantly mitigating the topological inaccuracies introduced by quantization error.

Problem

• CAD generation
• entity selection
• topological errors
• command sequences
• B-rep

Innovation

• Pointer-based selection
• B-Rep integration
• LLM-based CAD generation
• Geometric entity selection
• Topological error reduction