Self-Supervised Multi-Part Articulated Objects Modeling via Deformable Gaussian Splatting and Progressive Primitive Segmentation

2025-06-11

AI Summary

To address the challenge of 3D modeling for multi-part articulated objects in fully unsupervised, label-free settings, this paper proposes DeGSS: a framework that uniformly represents objects as state-driven deformable 3D Gaussian fields and achieves end-to-end, unsupervised rigid part disentanglement via self-supervised deformation learning. Key contributions include: (i) the first unsupervised joint geometric–kinematic modeling of movable parts; (ii) deformation trajectory-guided coarse-to-fine progressive part segmentation; and (iii) support for part-level reconstruction and precise kinematic relationship inference. The method integrates deformable Gaussian splatting, synthetic data augmentation (leveraging the newly introduced RS-Art real-simulation paired dataset), and self-supervised optimization. Evaluated on extended PartNet-Mobility and RS-Art benchmarks, DeGSS achieves state-of-the-art performance in geometric accuracy, motion consistency, and cross-state generalization.

Technology Category

• Computer Vision: Motion & Tracking
• Machine Learning: Unsupervised & Self-Supervised Learning
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Articulated objects are ubiquitous in everyday life, and accurate 3D representations of their geometry and motion are critical for numerous applications. However, in the absence of human annotation, existing approaches still struggle to build a unified representation for objects that contain multiple movable parts. We introduce DeGSS, a unified framework that encodes articulated objects as deformable 3D Gaussian fields, embedding geometry, appearance, and motion in one compact representation. Each interaction state is modeled as a smooth deformation of a shared field, and the resulting deformation trajectories guide a progressive coarse-to-fine part segmentation that identifies distinct rigid components, all in an unsupervised manner. The refined field provides a spatially continuous, fully decoupled description of every part, supporting part-level reconstruction and precise modeling of their kinematic relationships. To evaluate generalization and realism, we enlarge the synthetic PartNet-Mobility benchmark and release RS-Art, a real-to-sim dataset that pairs RGB captures with accurately reverse-engineered 3D models. Extensive experiments demonstrate that our method outperforms existing methods in both accuracy and stability.

Problem

• Modeling multi-part articulated objects without human annotation
• Unifying geometry, appearance, and motion in one representation
• Achieving unsupervised part segmentation and kinematic modeling

Innovation

• Deformable 3D Gaussian fields for unified representation
• Progressive primitive segmentation without supervision
• Spatially continuous part-level kinematic modeling