Instruct-Particulate: Scaling Feed-Forward 3D Object Articulation with Kinematic Control

2026-06-12

AI Summary

Existing methods struggle to generalize to unseen categories or AI-generated articulated 3D objects due to scarce annotated data. This work proposes integrating instructional kinematic specifications—comprising part descriptions, connectivity relationships, and joint types—into the reconstruction pipeline, enabling, for the first time, specification-guided end-to-end motion structure prediction. By constructing a heterogeneous 3D dataset of over 150,000 instances and leveraging vision-language models to automatically generate specifications during inference, the approach supports multi-granularity annotations and effectively unifies heterogeneous data sources. Experiments demonstrate that the method substantially improves generalization across object categories and AI-generated meshes, achieving high-quality reconstruction of articulated 3D assets from a single input image.

Abstract

Reconstructing articulated 3D objects is important for animation, gaming, and robotic simulations. Recent neural networks can estimate the articulated structure of 3D objects, but their generalization remains limited by the scarcity of annotated data for this task. To address this gap, we introduce Instruct-Particulate, a model that takes a 3D mesh together with a target kinematic specification, including part descriptions, connectivity, joint types, and optional point prompts, and predicts the corresponding kinematic part segmentation and joint motion parameters. The kinematic specification disambiguates the task and allows the model to target annotations of different granularity, thereby making it possible to use more abundant heterogeneous training data. At test time, the kinematic specification can be obtained automatically from large-scale vision-language models, so the model can be applied to any input mesh. To train our model at scale, we construct a heterogeneous dataset of more than 150,000 articulated 3D objects, extending existing publicly available collections with data obtained by partially labelling other 3D models (monolithic or already decomposed into parts) with kinematic labels by means of vision-language models. Experiments show that our model generalizes better across categories and to AI-generated meshes, enabling articulated asset reconstruction from real-world images via image-to-3D models.

Problem

• articulated 3D objects
• kinematic control
• data scarcity
• generalization
• 3D reconstruction

Innovation

• kinematic specification
• articulated 3D reconstruction
• vision-language models
• heterogeneous training data
• part segmentation