PWM-ArtGen: Part World Model for Articulated Object Generation

2026-07-02

AI Summary

This work addresses the challenge of generating articulated 3D objects from a single image, where accurately inferring kinematic structures remains difficult due to insufficient static visual cues, error propagation in two-stage pipelines, and scarcity of motion-labeled data. To overcome these limitations, we propose PWM-ArtGen, a unified part-based world model that, for the first time, formulates articulated objects as dynamic systems. By coupling image diffusion with action diffusion, our approach enables joint training of visual and action branches without requiring explicit kinematic annotations. We introduce a dual-diffusion architecture with independent timesteps and construct a large-scale dataset of part-level image pairs to support this framework. Experiments demonstrate that our method significantly outperforms existing baselines in static pose generation and exhibits strong zero-shot generalization, effectively handling complex out-of-distribution real-world objects.

Abstract

The key challenge in articulated 3D object generation from a single image is accurately predicting the underlying kinematic structure. Existing methods either infer kinematic parameters directly from a static image that lacks dynamic part-level kinematic relationships, or estimate parameters from visual dynamics generated from a single image, which is prone to accumulated errors of two steps. Moreover, the limited scale and diversity of existing annotated datasets further hinder generalization to complex, real-world objects. To overcome these limitations, we propose to learn the joint distribution of visual dynamics and kinematic parameters. Recognizing that articulated objects can be formulated as dynamic systems, we propose a unified Part World Model called PWM-ArtGen. To leverage unannotated data, this model couples action diffusion and image diffusion with independent diffusion timesteps, which enables visual branch co-training. We further curate a photorealistic dataset of 19.7k part-level image pairs without kinematic annotations, to support co-training. Experiments demonstrate that PWM-ArtGen substantially outperforms existing baselines in the resting state and exhibits strong zero-shot generalization to out-of-distribution objects.

Problem

• articulated object generation
• kinematic structure prediction
• single-image 3D reconstruction
• visual dynamics
• zero-shot generalization

Innovation

• Part World Model
• kinematic structure
• diffusion coupling
• unannotated co-training
• articulated object generation