PRISM: Probabilistic Representation for Integrated Shape Modeling and Generation

2025-04-06

AI Summary

Addressing the challenge of jointly preserving geometric complexity and semantic consistency in part-level 3D shape generation—while inadequately modeling variable part counts—this paper proposes a composable probabilistic representation framework integrating Statistical Shape Models (SSMs) and Gaussian Mixture Models (GMMs). We introduce the first unified architecture that synergistically combines classification-based diffusion models, SSMs, and GMMs to jointly model part-wise geometric deformations and semantic distributions in a continuous latent space. The framework enables seamless generation, reconstruction, and fine-grained semantic-driven editing of shapes with arbitrary numbers of parts. Evaluated on multiple benchmark datasets, our method achieves significant improvements: a 21.3% reduction in shape reconstruction error and a 36.7% increase in part editing success rate, while maintaining high fidelity, diversity, and structural consistency of generated outputs.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems

Abstract

Despite the advancements in 3D full-shape generation, accurately modeling complex geometries and semantics of shape parts remains a significant challenge, particularly for shapes with varying numbers of parts. Current methods struggle to effectively integrate the contextual and structural information of 3D shapes into their generative processes. We address these limitations with PRISM, a novel compositional approach for 3D shape generation that integrates categorical diffusion models with Statistical Shape Models (SSM) and Gaussian Mixture Models (GMM). Our method employs compositional SSMs to capture part-level geometric variations and uses GMM to represent part semantics in a continuous space. This integration enables both high fidelity and diversity in generated shapes while preserving structural coherence. Through extensive experiments on shape generation and manipulation tasks, we demonstrate that our approach significantly outperforms previous methods in both quality and controllability of part-level operations. Our code will be made publicly available.

Problem

• Modeling complex 3D shapes with varying part counts accurately
• Integrating structural and contextual information in shape generation
• Ensuring part-level fidelity and diversity in generated shapes

Innovation

• Integrates categorical diffusion with SSM and GMM
• Uses compositional SSMs for part-level variations
• Represents part semantics via GMM in continuous space