Harnessing Generative Image Models for Training-Free Primitive Shape Abstraction

2026-07-06

AI Summary

This work proposes a training-free method for abstracting 3D objects into compact sets of superquadric primitives, applicable to objects of arbitrary categories. By leveraging multi-view rendering and vision-language models, the approach parses semantic parts through prompt engineering to guide generative image models in producing color-coded part segmentation masks. These masks are then back-projected onto the 3D surface, and individual superquadrics are optimized to fit each part. As the first fully training-free, category-agnostic, and orientation-invariant primitive abstraction framework, its performance inherently improves with advances in generative models. Experiments on the HumanPrim and Toys4K datasets demonstrate that using only 5–9 primitives achieves the lowest Chamfer distance, indicating that the current accuracy bottleneck lies in part segmentation rather than the primitive fitting capability.

Abstract

Representing 3D shapes as compact sets of geometric primitives is fundamental to robotics, simulation, and scene understanding. Generative image models trained at scale have recently emerged as generalist visual learners that can identify and segment object parts directly in the image domain, across arbitrary categories and without task-specific training. Adapting such models to downstream tasks typically requires fine-tuning; we ask whether their pretrained capability can instead be harnessed directly, without any training, and answer affirmatively with a training-free harness. Our pipeline renders multi-view images of a 3D object, uses a vision-language model to analyze its semantic parts, prompts a generative image model to paint a color-coded part segmentation mask, reprojects it onto the geometry, and fits a superquadric primitive to each part via parameter optimization. The approach contains no learned parameters: it is category-agnostic and orientation-invariant, properties that previous learning-based models struggled with. Its accuracy ceiling rises with future generative-model improvements, which we confirm with a ground-truth segmentation study showing that part segmentation, not primitive fitting, is the current accuracy bottleneck. On HumanPrim and Toys4K, our method achieves the lowest Chamfer distance among all evaluated methods, using 5--9 primitives per object on average.

Problem

• 3D shape abstraction
• geometric primitives
• training-free
• generative image models
• part segmentation

Innovation

• training-free
• generative image models
• primitive shape abstraction
• superquadric fitting
• vision-language prompting