3D-DLP: Self-Supervised 3D Object-Centric Scene Representation Learning

2026-06-17

AI Summary

This work addresses the challenge of learning interpretable and structured 3D scene representations from RGB-D or voxel data under fully unsupervised conditions, with the goal of supporting downstream tasks such as robotic manipulation. To this end, the authors propose a self-supervised method that decomposes a scene into a set of implicit particles, each endowed with disentangled attributes—namely 3D keypoints, size, and appearance—and generates corresponding segmentation masks through end-to-end reconstruction. This approach is the first to extend object-centric representations to 3D while enabling fully self-supervised training. The learned particles exhibit both interpretability and controllability, facilitating scene editing and efficient inference. Experiments demonstrate that the proposed representation significantly outperforms baseline methods that either rely on dense 3D inputs or lack structural priors, leading to improved performance in robotic manipulation tasks on both simulated and real-world data.

Abstract

We introduce 3D-DLP, a self-supervised object-centric representation learning model that decomposes scene-level RGB-D or voxel observations into a set of 3D latent particles. Building on the Deep Latent Particles (DLP) framework, each particle encodes disentangled attributes, including 3D keypoint position, bounding box dimensions, and appearance features, and represents a distinct entity in the scene. The model learns interpretable per-particle segmentation maps through an end-to-end self-supervised reconstruction objective. We demonstrate on both simulated and real-world datasets that the learned latent space is interpretable and controllable: by manipulating particle positions and decoding, we can generate novel scene configurations. Furthermore, we show that leveraging these compact 3D latent particles for downstream robotic manipulation improves performance over baselines that either lack explicit 3D information or rely on memory-intensive dense 3D inputs without object-centric structure. Code and videos are available at https://eubooks3003.github.io/3d-dlp.

Problem

• 3D object-centric representation
• self-supervised learning
• scene decomposition
• latent particles
• robotic manipulation

Innovation

• self-supervised
• object-centric
• 3D scene representation
• latent particles
• robotic manipulation