SPLATART: Articulated Gaussian Splatting with Estimated Object Structure

2025-06-13

AI Summary

Representing complex articulated objects (e.g., robotic arms, cabinet doors) in 3D remains challenging due to the need to jointly model geometry, appearance, part segmentation, connectivity, and high-dimensional joint parameters—especially under high degrees of freedom (≥7) or deep kinematic trees. Method: We propose the first differentiable Gaussian splatting representation that decouples part segmentation from joint parameter estimation. Our approach integrates image-space segmentation supervision, pose constraints, and explicit kinematic tree modeling to enable posterior learning of joint parameters. Results: Evaluated on the synthetic Paris dataset, our method achieves accurate reconstruction with only sparse segmentation supervision in real-world scenes. It is the first to generalize successfully to complex structures such as deep serial-chain manipulators. The framework significantly improves both geometric fidelity and kinematic accuracy in joint object representation, while enhancing generalization across diverse articulated structures.

Technology Category

• Computer Vision: Representation Learning for Vision
• Intelligent Robots: Manipulation
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Representing articulated objects remains a difficult problem within the field of robotics. Objects such as pliers, clamps, or cabinets require representations that capture not only geometry and color information, but also part seperation, connectivity, and joint parametrization. Furthermore, learning these representations becomes even more difficult with each additional degree of freedom. Complex articulated objects such as robot arms may have seven or more degrees of freedom, and the depth of their kinematic tree may be notably greater than the tools, drawers, and cabinets that are the typical subjects of articulated object research. To address these concerns, we introduce SPLATART - a pipeline for learning Gaussian splat representations of articulated objects from posed images, of which a subset contains image space part segmentations. SPLATART disentangles the part separation task from the articulation estimation task, allowing for post-facto determination of joint estimation and representation of articulated objects with deeper kinematic trees than previously exhibited. In this work, we present data on the SPLATART pipeline as applied to the syntheic Paris dataset objects, and qualitative results on a real-world object under spare segmentation supervision. We additionally present on articulated serial chain manipulators to demonstrate usage on deeper kinematic tree structures.

Problem

• Representing articulated objects with geometry and joint details
• Learning complex articulated objects with multiple degrees of freedom
• Disentangling part separation from articulation estimation for deeper kinematic trees

Innovation

• Uses Gaussian splatting for articulated objects
• Disentangles part separation from articulation estimation
• Handles deeper kinematic trees than previous methods