StrucADT: Generating Structure-controlled 3D Point Clouds with Adjacency Diffusion Transformer

2025-09-28

AI Summary

Existing 3D point cloud generation methods lack explicit structural controllability, hindering user-customized design. To address this, we propose the first structure-aware control paradigm based on part existence and adjacency relations, introducing StructureGraph—a unified topological representation of part-level structure. We further develop an end-to-end controllable generation framework comprising three core components: (1) StructureGraphNet, which extracts structure-aware features; (2) a conditional continuous normalizing flow (cCNF) prior that models the latent distribution conditioned on the StructureGraph; and (3) a conditional diffusion Transformer for high-fidelity point cloud synthesis. Crucially, our method enforces strict structural consistency throughout generation. Evaluated on ShapeNet, it achieves state-of-the-art performance in both fidelity and diversity, while enabling precise, user-specified structural layouts—supporting controllable, high-quality, and structurally faithful point cloud generation.

Technology Category

• Machine Learning: Structured Learning
• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Web Mining and Content Analysis: Bridging structured and unstructured data

Abstract

In the field of 3D point cloud generation, numerous 3D generative models have demonstrated the ability to generate diverse and realistic 3D shapes. However, the majority of these approaches struggle to generate controllable 3D point cloud shapes that meet user-specific requirements, hindering the large-scale application of 3D point cloud generation. To address the challenge of lacking control in 3D point cloud generation, we are the first to propose controlling the generation of point clouds by shape structures that comprise part existences and part adjacency relationships. We manually annotate the adjacency relationships between the segmented parts of point cloud shapes, thereby constructing a StructureGraph representation. Based on this StructureGraph representation, we introduce StrucADT, a novel structure-controllable point cloud generation model, which consists of StructureGraphNet module to extract structure-aware latent features, cCNF Prior module to learn the distribution of the latent features controlled by the part adjacency, and Diffusion Transformer module conditioned on the latent features and part adjacency to generate structure-consistent point cloud shapes. Experimental results demonstrate that our structure-controllable 3D point cloud generation method produces high-quality and diverse point cloud shapes, enabling the generation of controllable point clouds based on user-specified shape structures and achieving state-of-the-art performance in controllable point cloud generation on the ShapeNet dataset.

Problem

• Generating controllable 3D point clouds meeting user-specific structural requirements
• Addressing lack of control over part existences and adjacency relationships
• Enabling structure-consistent generation through annotated part adjacency relationships

Innovation

• StructureGraph representation for shape control
• StrucADT model with three specialized modules
• Adjacency-conditioned diffusion transformer for generation